Vespene is a modern, streamlined build and self-service automation platform. Vespene is designed to combat chaos in complex software development and operations environments.
Our mission is simple: get great people together to build the ultimate system we all want to use.
Vespene is a horizontally-scalable Python application, using Django and PostgreSQL. Each node in a Vespene cluster runs a copy of the web code and any number of backend build “worker” processes, all of which share the database. Users can connect to any node in the cluster.
While equally usable for small shops, Vespene was inspired by the needs of large organizations with hundreds of microservice projects in dozens of languages.
When organizations start to have too many projects, they need an easy way to share values and code snippets between those projects. To support this we have Jinja2 templating of build scripts via Variables built-in. Variables can be defined across multiple hierarchies of objects throughout Vespene, and are also directly consumable by loading “vespene.json” from the build root, which interfaces nicely with any application that can read YAML or JSON.
Automation, “DevOps”, Etc¶
Projects in Vespene don’t need to just represent source code. Projects can also launch automation scripts or software of any kind, whether being pulled from source control or just scripts defined in Vespene. These jobs can include security scans, cloud topology changes, database backups, launching functional tests, and more.
While not all automation tools are SSH-powered, many are. Vespene can memorize important SSH Keys and use them on your behalf, using built-in support for ssh-agent.
Helping with the self-service magic, projects can ask interactive questions before they are launched with Launch Questions, providing unprivileged users a simple interface to invoke specific tasks without needing full shell access. These questions can be drop-downs, multiple choice, or fill in the blank.
Vespene also contains pluggable Autoscaling technology, easily backed by tools such as AWS CloudFormation and Terraform - so you can always have exactly the right amount of build resources available and cut down on cloud costs.
Vespene features Pipelines to construct complex CI/CD topologies - successful builds can trigger other builds, in serial or parallel. Pipelines in Vespene are easily graphically configured, or membership can be defined by declarative .vespene files in YAML. There is no custom DSL (“Domain Specific Language”) to learn and debug. Use pipelines to implement fully automated deployment to production, or just stop at staging - whatever you feel comfortable with.
Another key inspiration for Vespene was frustration with setup, configuration, and maintenance of other CI/CD platforms.
Vespene is also straightforward to Setup and maintain. GitHub organizations can be sucked in automatically, and configured with “.vespene” files (see Importing Organizations). From an admin perspective, there are no message queues or clunky leader election clusters to keep running, and every node in the cluster has the same configuration.
Finally, Vespene has an exceptionally concise and extensible codebase. Nearly all policy in Vespene is implemented in Plugins, and more areas for pluggability will be added over time.
Vespene aims to be usable on the master branch and welcomes feedback, testing, new features, and ideas of all kinds. Database migrations will always preserve your data, even during development.
Officially, Vespene is in beta as it just came out! We are shooting for our first stable release in early January of 2019 and are likely going to do releases about every three months thereafter.
Whether you have a classic deployment setup or are going all-in on microservices and need to tame the sprawl, we hope Vespene will be the build system for you. If it’s not yet, we’re always improving and on the lookout for new ideas. Things are new, flexible, and everything is up for discussion.