Syndicate Guides

Documentation for Users, Developers, and Administrators

© 2017. All rights reserved.

A Syndicate CI/CD Strategy

Syndicate Continuous Integration and Continuous Delivery Strategy

Our software development strategy is based on concepts established under the “Continuous Integration” and “Continuous Delivery” (CI/CD) models. This approach is based on the practice that developers integrate code into a shared repository as frequently as possible. The regular and short integration cycles generally lead to improved quality of software and a significantly reduced time to deliver it. Small and frequent changes enable us to quickly identify and make corrections or improvements. And with integrated automation, the process of rebuilding, deploying, testing, and analyzing is executed immediately after every merge.

These are some of the key components of our CI/CD environment, which will be described in more detail below:


DockerLogo

Integration with Jenkins

We utilize a build server running Jenkins to automate the build/delivery process. The Jenkins server is also integrated with our testing and analysis framework. When a code change is merged, the build server detects the change and automatically rebuilds the associated Syndicate binaries and packages. Within a few moments of a change, the automated testing begins and users can begin installing the newest Syndicate packages! Literally thousands of tests are performed and – when complete – the results of the tests are posted to the Jenkins dashboard and developers are also instantly notified via “Slack”. To assist with the analysis process, tests also include valgrind/memcheck reports which are integrated directly with Jenkins and are visible from the Jenkins dashboard. We include profiling data as part of the analysis output, thus making it possible for developers to not only quickly find errors or potential memory leaks, but to also find potential timing related issues or see basic flow diagrams / call graphs associated with each test!


DockerLogo

Docker and Ansible

Docker is an essential part of our CI/CD development environment. It provides the ability for developers to run Syndicate in light-weight, isolated “containers” that are identical between each developer. We also use Ansible in conjunction with Docker, thus providing the ability to quickly install/rebuild fresh testing and development environments on our test/demo servers - these are automatically configured to be identical to the docker containers. In our environment, the syndicate-ci repository is responsible for creating a Docker based test environment for the Jenkins server, whereas our syndicate-deploy repository can be used to deploy the same test environment in either Docker containers or actual hardware, for the purpose of testing or development. Using this strategy provides the ability for a new developer to begin developing or testing in literally minutes.


DockerLogo

Automated testing framework with integrated analysis/debugging

This environment also includes a unique testing framework consisting of thousands of tests, all of which are tracked and reported to our Jenkins server. Although we do not develop strictly under a “test-driven” environment, our philosophy is to create tests for each feature, risk, and requirement of Syndicate. For this reason, our testing framework (syndicate-tests) was designed to be versatile, extensible, and easy to use, thus allowing developers to quickly create new tests that meet our CI testing standards. The testing framework uses YAML based test files which further simplifies the creation and design of each test. Integrated within the framework, each test can optionally be run with various profiling tools (valgrind/memcheck, callgrind, sprof, oprofile, etc). The test framework knows how to process the resulting files by organizing them, reformatting them, or even producing helpful plots. It is standard practice to run tests locally prior to merging code changes to the mainline. For this reason, the test framework is also available and installed from the syndicate-deploy repository, allowing developers to test or make changes directly from within docker containers. Furthermore, analysis reports are not only integrated within Jenkins but they are optionally available during standalone testing. Another feature that is quite unique to our development environment is the ability to develop and debug directly from the test framework. For example, imagine a developer encounters an issue corresponding to a specific test. And according to the debug logs, the line numbers corresponding to the issue may also be available. With this information, the developer can simply instruct the specific test to launch a debugger and break in the exact area(s) of interest.


DockerLogo

Collaboration Tools

A vital part of continuous integration comes from the developers ability to work together, share information, to collaborate, and to receive notifications about the state of the system. We use “Slack” for our collaborative needs. Not only is much of our communication done exclusively through slack, which is helpful for maintaining records and a history of technical conversations; but our Jenkins server also posts statuses regarding the state of each repository along with links to test results/artifacts.


Watch the Syndicate CI/CD demonstration video


Current and Future CI/CD Efforts

In the near future, we plan to expand support in the following areas: