Introduction

Update: Most of the necessary functionality was provided in [1187].

For the primary motivation behind this project, see Object Collection. The central Puppet daemon (called puppetmasterd) needs to store the elements generated by each client so that those elements can be queried against and collected. It is also critical that this storage be configurable so that multiple Puppet servers can use the same store; otherwise it will be impossible to horizontally scale the Puppet service.

Object collection is already spoken about enough in the Object Collection page, but it's worth mentioning querying and reporting. Because the Puppet servers are responsible for configuring the entire network, it would be hugely beneficial if the compiled elements could be queried against. It is not sufficient to make the text-based configuration queryable because it only makes sense to query the compiled configuration. One of the roles of this database backend will be to store all of the compiled elements so that they are available for reporting (although there are no concrete plans yet to provide such reporting mechanisms, they should be relatively easy).

The real motivation behind this, though, is object collection.

Implementation

I really expect this imlementation to be relatively straightforward, done using ActiveRecord from Ruby on Rails, just because it provides simple database abstraction and supports Sqlite out of the box, which is an important feature for the standard case where there's only one Puppet server.

The storage itself should be pretty easy -- each element gets stored with a type, a name, and a set of parameters, plus a boolean to mark whether it's collectable or not. The first stage is just supporting direct access from puppetmasterd without worrying about reporting or anything, so that should not be all that complicated.

Client-side Databases

At some point the client also needs to have a database for storage, although it likely will never make sense to support much more than Sqlite. The client currently uses YAML to cache the data, but the cache file is relatively brittle and it doesn't support concurrent access by multiple daemons.