As you may know, Command Prompt, Inc. develops and maintains PostgreSQL snap packages as a service to community. If you used it, you also know that unfortunately it is not yet a drop-in replacement for DEB builds distributed via PGDG APT repository.

PostgreSQL is one of the major open source projects out there that is extremely popular with all sorts of crowds: from enthusiasts to unicorn startups. And as the Snapcraft ecosystem matures more and more, a fully-functional PostgreSQL snap package becomes absolutely necessary for reasons ranging from developers being able to easily run PostgreSQL in strictly isolated test environments, to providing more complex software solutions that build on PostgreSQL (e.g. Travis CI people expressly stated they were very much interested in getting their hands on it) to Canonical running their infrastructure on PostgreSQL packaged as a snap.

Canonical, the company behind Ubuntu and the snap packages, invited us to Snapcraft Summit, a 3-day hackathon that took place in central London, United Kingdom in the beginning of November. It is a forward-thinking software workshop attended by major software vendors and Snapcraft engineers working at every level of the stack.

The lineup was excitingly diverse. Among the projects that I can remember there were representatives of KDE, Transmission, Ruby, Travis CI, keepalived, Postman, Volumio as well as Microsoft, IBM and others.

The goal was to get all together in one big room, sit down with Snapcraft and snapd engineers and hack together on snap packages.

In our case, the PostgreSQL snap package.

We kicked off day 1 with basics of how snap packages work and how to create them, as some of the attendees had no prior experience of working with Snapcraft and snap packages. Our case, in some ways similar to keepalived, was all about adding new features as well as general polish.

I worked primarily with Evan Dandrea, an Engineering Manager at Canonical, responsible for the Snapcraft ecosystem. After the initial orientation, we discussed and laid out a plan of changes:

• move to tracks and use a single snap package to distribute all major and bugfix versions of PostgreSQL
• confirm parallel installation is possible, which would allow to run multiple instances of PostgreSQL, including exact same version of PostgreSQL.
• set up global aliases that would allow to call binaries in a familiar way (e.g. psql vs postgresql.psql)
• turn PostgreSQL snap package into a proper daemon (specifically, systemd service)
• and see if we can do away with wrappers, that we had to use to get PostgreSQL working correctly as a snap.

In fact, since the PostgreSQL snap package already exists and works, certain limitations and oddities notwithstanding, we spent quite a bit of time just wrapping our heads around where we are at now, where we want to go with all this and how to get there.

At some points we had various Snapcraft engineers and CTO of the Snapcraft team, Gustavo Niemeyer, join the discussion. We discussed in detail how people use PostgreSQL, what their expectations are of what they can do with it as a snap package and many other things. Eventually, we came out with an understanding of both the needs of PostgreSQL and people who use it as well as problems that Snapcraft aims to resolve in software packaging world. One of the biggest issues which is, lots and lots of outdated software that creates unnecessary security problems at a very large scale. Canonical’s hope apparently is to get it under control in a way that helps less proficient users maintain higher levels of security of their systems while allowing more experienced ones to decide how exactly they want to manage their software (e.g. whether to update automatically a PostgreSQL snap package installed in the system).

The results of the 3-day hackathon are a mixed bag of success and realization that snapd and snapcraft need to improve in certain areas to support typical use cases of PostgreSQL and traditional RDMS in general.

Unfortunately, even working shoulder-to-shoulder with core snapd developers we were not able to find a quick fix that would allow us to run PostgreSQL as a daemon. Similarly, getting rid of wrappers that are currently necessary to make a locale available in the strictly confined snap environment is not an easy feat either. On the bright side, Snapcraft team had a chance to consider these issues and promised to resolve them in the near future.

That said, we successfully moved to tracks which allows us to have a single snap package that can be used to install any major or bugfix version of PostgreSQL. We also tested and confirmed that parallel installations work with PostgreSQL. This particular feature requires latest version of snapd (> 2.36) and allows one to run multiple instances of PostgreSQL on the same system, including multiple instances of the exact same version of PostgreSQL. Aliases, another useful feature, are also available now. Combined with tracks and single package design, this makes sure that psql, pg_dump and other common utilities can not only be called in a familiar way (i.e. no need for snap package name prefix, like postgresql.pg_dump) but also ensures that psql, pg_dump and other aliases always point to the most recent version of these tools.

Throughout the event, from welcome drinks in a local pub in Southwark to the farewell dinner and beers (and a cup of tea in my case), I couldn’t help but think all the time how welcoming and friendly everyone was. The Snapcraft team felt just as if I had been part of it for many years. All great people. Incredibly passionate about what they do. Hilarious jokes bursting here and there all the time. Good laughs. Good food. Good times.

Most excitingly, you sit in a room with the very same people who created the technology that you use to package PostgreSQL and you have this privilege of asking them directly any questions you might have. You can pull in anyone to help you figure out this or that problem. It was an incredibly empowering feeling and experience. But the truth is, at the end of this 3-day massively multivendor offline collaboration we went back to our homes having made plans to continue working together on snap packages.

In fact, the very next week after the summit I joined what would otherwise be an internal Snapcraft team’s online meeting to discuss Canonical’s production requirements for PostgreSQL. Same engineers we worked shoulder-to-shoulder with carry on today with conversations in Snapcraft forum we had started during hackathon.

I hope Canonical gained just as much valuable feedback and insight from this collaboration. It certainly felt like they were there to eagerly listen and I made sure they receive plenty of it. It was kind of eerie in most positive way, because you don’t really get this feeling in offline world that often.

And that, in my opinion is why Snapcraft Summit is such a great event that really helps develop strong bonds in open source community.