Archive for the ‘Puppet’ Category

Zynga uses Puppet to manage configuration of FarmVille’s web farm

Wednesday, February 10th, 2010

In an interview with Zynga’s Luke Rajlich on the High Scalability website one interesting tidbit in the post was their use of Puppet to manage configuration of the web farm for their extremely popular FarmVille game. If you aren’t familiar with Zynga, they are one of the gaming giants on Facebook, offering such mega hits as Mafia Wars, FarmVille, Zynga Poker, and many others. FarmVille alone has 75 million players 28 million of which play daily. In addition at peak traffic times Luke indicated that “roughly 3 Gigabits/sec of traffic go between FarmVille and Facebook while our caching cluster serves another 1.5 Gigabits/sec to the application”.

The article gives some great insights into the challenges of running an online game and how they interact with Facebook. At the end of the post Luke wrote about how he manages their web farm:

To help manage and monitor FarmVille’s web farm, we utilize a number of open source monitoring and management tools. We use nagios for alerting, munin for monitoring, and puppet for configuration. We heavily utilize internal stats systems to track performance of the services the application uses, such as Facebook, DB, and Memcache. Additionally, when we see performance degradation, we profile a request’s IO events on a sampled basis.

It’s good to see Puppet being used in some of the world’s most demanding environment where efficiency, reliability and predictability are essential.

Reductive Labs announces Puppet training dates for London, New York, and Nuremberg

Monday, February 8th, 2010

Puppet Training is popular apparently! Due to demand, we’ve scheduled 3 public Puppet training courses in NY, London, and Germany. You can register and get more information about the training at this link. If you have any questions please contact Scott Campbell.

Location & Dates

Becoming a Puppet Master – 3 Days

Puppet Training consists of 3 days of hands-on training performed by a Reductive Labs Puppet professional. Attendees will be taught the principles and best practices of Puppet in a series of lectures and labs.This training is ideal for those who want a Puppet jumpstart. Newer members at an organization already using Puppet, or experienced sysadmins wanting to bring Puppet into their team will get everything they need to deploy solutions.

Topics covered include:

  • Configuring Puppet and Puppetmaster
  • Resource Types and the Resource Abstraction Layer
  • Virtual Resources, Exported Resources and Stored Configs
  • Meta-parameters, Dependencies and Events
  • Classes, Modules and Definitions
  • Tags and Environments
  • Puppet Language Patterns and Best Practices

Puppet Developer Curriculum – 2 Days (NY & London Only)

This is an advanced course for those Puppet users who are interested in developing skills and learning best practices for creating their own custom Resource Types and Modules.

  • Introduction to Ruby for Puppet
  • Advanced Function and Fact development
  • Resource Type and Provider development
  • Testing practices and RSpec for Puppet

Looking forward to seeing you there!

Vote for the winner of the T-Shirt tagline contest

Thursday, February 4th, 2010

Last month we kicked off a contest to design the new tagline for our next t-shirt and now is the time for you to tell us who should win. We received a bunch of great entries and have narrowed it down to the following 8 finalists. Please vote for your favorite and decide the winner.

The person who submitted the winning entry will receive a BugBundle from Buglabs. It’s in your hands to determine who wins.



Michael DeHaan, Creator and Community Lead for Cobbler, Joins Reductive Labs as Product Manager for Puppet

Tuesday, February 2nd, 2010

Michael DeHaan joins Reductive Labs as Product Manager for PuppetReductive Labs is excited to announce the hiring of Michael DeHaan as Product Manager for Puppet. Michael will drive product strategy, roadmaps and community engagement for Puppet. Michael previously was the creator and architect of Cobbler at Red Hat as well as the community lead for that product.

Michael brings a strong background in open source software and community development to Reductive Labs. As the creator and community lead for Cobbler at Red Hat, Michael oversaw the growth of Cobbler, which is now used in thousands of datacenters across the world — including in the financial industry, hosting companies, render farms, grids, and universities. Cobbler is well known in the Enterprise Linux and Fedora space as the OS provisioning tool of choice for rapid deployment in medium to large-scale environments. It is very frequently used in conjunction with Puppet to maximize flexibility and efficiency in rollouts of new machines, whether physical or virtual.

In addition to the growth of the use of Cobbler, Michael guided tremendous growth of the contributing community. The Cobbler project has had over 80 code contributors and many more community members that assist with testing, advocacy, ideas, and support.

Michael is a published contributor to Red Hat Magazine and has presented at such events as Red Hat Summit, Red Hat Cloud Forum, the Fedora Users and Developers Conference, HP Tech Forum, and local software events. Michael is also a contributor to over 50 US patent applications in the area of configuration management and datacenter automation.

“Puppet has accomplished something few open source projects achieve — not only has Reductive Labs built a best of breed configuration management platform, it also has created a large and vibrant community of users and contributors that help guide its development,” said Michael about why he was joining Reductive Labs. “The future for Puppet’s ecosystem is extremely promising, and I look forward to helping it evolve and grow further in the years to come. Whether we are talking about cloud architectures, virtualization, grid, or classical server rollouts — as datacenter application deployment gets more complex, Puppet is around to help make the complicated simple and the impossible possible. For me, this is really one of the most exciting spaces in technology to be in, because not only can you help users all over the world solve their management challenges, but you also get to drive the forefront of computing.”

“We couldn’t be happier to add someone with the open source development and community credentials of Michael to our team,” said Luke Kanies, Founder and creator of Reductive Labs and Puppet. “Michael will help us identify opportunities to enhance the value of Puppet and engage further with our already strong and passionate community. We look forward to Michael helping guide the future of Puppet.”

Case Study: Sun Microsystems uses Puppet to accelerate system updates and ensure consistent configurations across their web server architecture

Tuesday, January 26th, 2010

MartinWe are happy to make a new case study available from Sun Microsystems on how they are benefiting from their use of Puppet to manage their web server architecture. Martin Englund, a lead engineer at Sun Microsystems, describes how he uses Puppet to greatly accelerate system updates for the servers under his control as well as ensure a consistent environment across all his servers for his web developers.

Martin went on to say:

“With Puppet I don’t have to worry anymore. Once I have written and deployed the profiles I can count on Puppet ensuring timely updates and consistent configurations across all my systems. More than anything Puppet saves me time that I simply can’t afford to lose in supporting my data centers.”

Among the key benefits that Sun Microsystems has gained from using Puppet are:

  • System consistency
  • Improved effeciency
  • Met compliance standards
  • Increased visibility

Get the full details and download the case study by clicking below

Sun Microsystems Case Study

Concise writeup from administrator at Berkeley Lab Commons on why he chose Puppet over CFEngine

Monday, January 25th, 2010

I came across this link today describing a few reasons why James Welcher at Berkeley Lab Commons decided to use Puppet instead of CFEngine. He includes a link to an article from Luke Kanies as well as some good links for migrating from CFEngine to Puppet.

In his words, the reason he ultimately decided on Puppet hinged on its ability to handle more complex tasks:

Puppet seems to handle more complex tasks, at the possible expense of more complex configuration, at least initially, but it can then handle “higher-level” funtions, like package management. This will be important to us if we are doing cross-platform (Linux as well as FreeBSD) management.

Many of the people we speak with are comparing Puppet with CFEngine and ultimately looking at the advantages they can gain from a more powerful tool. In part, Puppet was developed by Luke Kanies to address many of the shortcomings of CFEngine and to provide better tools to system administrators.

Case Study: Los Alamos Uses Puppet to Gain Visibility into Their Deployed Mac OS X Environment

Tuesday, January 19th, 2010

Allan Marcus, Solutions Architect at Los Alamos National LaboratoryWe are happy to make a new case study available from Los Alamos National Laboratory. Allan Marcus, a Solutions Architect, at Los Alamos, walks through how he used Puppet to gain visibility and control of their large deployed Mac OS X environment and meet NIST 800-53 security standards. Among the key benefits that they realized were:

  • Enhanced Visibility
  • Improved Effeciency
  • Adherance to compliance standards
  • Accelerated troubleshooting

When speaking about the benefits he realized from using puppet Allan says:

“Prior to using Puppet, managing the Mac OS X systems in our network was a challenge. There was a real lack of visibility into both the number of Macs on the network and their configuration. Puppet has made a real difference to our administrators who were previously having to walk to each Mac and service it individually.”

You can download the case study here:

Los Alamos National Laboratory Case Study

Design the new Puppet t-shirt tagline and win a BUGbundle from Bug Labs

Wednesday, January 13th, 2010

Calling the Puppet community.

We are currently looking for a replacement for our current t-shirt tagline “Because SSH and a ‘for’ loop doesn’t cut it” and we are asking you to help. Contribute your suggested tagline and if choose your tagline in our next t-shirt we will send you a BUGbundle from bug labs.
Design Puppet T-Shirt tagline and win a BUGbundle from Bug Labs

Submit your suggestions in one of the following ways:

We look forward to seeing your ideas. We will choose a winner by the middle of February.

Deploying Puppet in Client-Server, Standalone, and Massively Scaled Environments

Tuesday, January 12th, 2010

A common misconception about Puppet is that it can only be used in client/server mode. Although this is the most common use case, it is actually just one of three common deployment practices.

Client/Server model:

Client/server deployment is the most common and feature rich way to run Puppet. For any environments where multiple hosts are managed, client/server deployment is usually the way to go.

This mode has two executables, puppetmasterd (server), and puppetd (client). Knowing the roles of these executables is important for understanding the differences between these three deployment practices.

  1. Client gathers local facts about its system using facter.
  2. Client initiates a request to the server requesting the latest version of its catalog(description of desired configuration state)
  3. Server compiles the configuration from source(manifests) into a catalog and returns it to the client.
  4. Client applies the catalog, resulting in configuration changes.

This deployment strategy has several advantages over the standalone puppet execution.

  • All configuration source is centrally stored and managed on the puppet server.
  • The client only receives the configuration information that it needs (and can’t see the information that doesn’t).
    - The client doesnt have access to the source code, just the compiled catalog that applies to it.
  • The server allows for more complicated management of nodes using an external node classifier (like the dashboard).
    - A node classifier allows assignment of classes and parameters to be handled by an external script. This allows information about how nodes are defined to be controlled by external data sources (think CMDB)
  • Uses certificates to ensure that only authorized clients can retrieve configuration.
  • Centralized function execution can simplify custom functions.

The main constraint of the client/server model of Puppet is that the server is CPU bound on catalog compilations. This can limit the scale at which puppet can be run on a single Puppet masters. The limiting factors of scale are:

  • Number of hosts per Puppet server.
  • Interval between client check-ins.

Many of these performance constraints have already been addressed in 0.25.x, and will continue to be improved upon as Puppet improves its ability to pre-compile catalogs

Single host:
One of the most overlooked features in Puppet is the puppet executable. This allows you to both compile and apply catalogs locally, removing the complexity of running in client/server mode.

This makes puppet a viable tool for application bootstrapping, even on one machine.

Executing:

#>puppet mymanifest.pp

will apply the configuration from the local source file mymanifest.pp to the local host. The puppet executable takes many of the same long options as puppet, namely:

  • –noop – allows you to see the effects of the compiled catalog without making any modifications to the local machine
  • –verbose, –debug – increased logging output
  • –modulepath – puppet standalone can use modules to organize and re-use code just like client server
  • –environment – standalone can also use multiple environments.

The puppet executable also reads configuration from the [puppet] section of puppet.conf

Running puppet on a single machine is not only a perfectly valid use case, it is also the best way to get started with the puppet language, and the best way to quickly develop and test new manifests.

Massively scalable deployments:

Some users have opted not to use the Puppet master at all for scaling reasons. Although we feel that we have addressed most of these scaling issues with performance improvements in .25.x, the PuppetMaster will always be the bottleneck in client/server mode.

For this deployment practice, users compile catalogs and apply configuration changes locally with the puppet executable. Code on all of the hosts is then kept in sync with a central repository using something like rsync.

This will always be the best way to scale Puppet deployments, but it suffers from a loss in security as well as ease of management.

As a follow up, I will give some examples of running puppet in standalone mode.

OSU’s Lance Albertson details reasons for switching to Puppet

Friday, January 8th, 2010

Came across this link with an interview of Lance Albertson of Oregon State University’s Open Source Lab talking about how they are managing their data center. Among the topics was a description of their reasons for switching from CFengine to Puppet. Here is what Lance had to say:

OSUOSL uses Cfengine for systems management but hopes to migrate onto Puppet. What are the primary advantages of Puppet?

Albertson: For one, Puppet has a real community behind it instead of a single person who gets to decide what to include for patches. Additionally, Puppet’s primary goal is managing a large installation of servers instead of being a research project at a university (which Cfengine is). Also the primary contributors of Puppet just moved to Portland and have offered help to us if we switch, so that is an advantage.

From a technical point of view, Puppet offers more flexibility and an ability to actually use real code to deal with tasks. Cfengine has its own syntax language, but it’s not really suited for complex tasks. Puppet also offers some inheritance in its rules, so we could write a module that just does Apache configuration and have a host inherit that code. Then we can offer our module to the world without worrying about revealing security issues.”

Glad to see a good example of the strength of the Puppet community being a key factor in the decision companies make in their choice of tools. Thanks to R.I. Pienaar (@ripienaar) for the pointer to this article.