123
 123

  2013-02-19 Tue

09:34 Puppet monitoring: how to monitor the success or failure of Puppet runs (1627 Bytes) » High Scalability

This is a guest post by LogicMonitor's Director of Tech Ops, Jesse Aukeman, about the different ways they're monitoring the success or failure of Puppet runs.

If you are like us, you are running some type of linux configuration management tool. The value of centralized configuration and deployment is well known and hard to overstate. Puppet is our tool of choice. It is powerful and works well for us, except when things don't go as planned. Failures of puppet can be innocuous and cosmetic, or they can cause production issues, for example when crucial updates do not get properly propagated.

Why?

In the most innocuous cases, the puppet agent craps out (we run puppet agent via cron). As nice as puppet is, we still need to goose it from time to time to get past some sort of network or host resource issue. A more dangerous case is when an administrator temporarily disables puppet runs on a host in order to perform some test or administrative task and then forgets to reenable it. In either case it’s easy to see how a host may stop receiving new puppet updates. The danger here is that this may not be noticed until that crucial update doesn't get pushed, production is impacted, and it’s the client who notices.

How to implement monitoring?

09:30 Sponsored Post: OLO, Amazon, Zoosk, aiCache, Teradata Aster, Aerospike, Percona, ScaleOut, New Relic, Logic Monitor, AppDynamics, ManageEngine, Site24x7 (6008 Bytes) » High Scalability

Who's Hiring?

  • OLO's food ordering platform powers some of the largest restaurant chains and feeds millions of consumers. We're looking for Senior C# Software Engineers and DevOps Engineers to help us scale our system. Apply here.
  • The AWS Relational Database Service (RDS) automates management of relational databases in the cloud. We have a wide variety of customers and are part of many mission-critical applications, like the ones built by the 2012 Obama re-election campaign. If you're interested in joining a fast-growing service and team, please send your resume to rds-jobs@amazon.com.
  • Hiring! Director of Site Operations at Zoosk.  We’re looking for an innovator. Someone who wants to take site operations along with a smart team of Sys Admins to the next level. This is a very hands-on leadership role in a high-availability production environment. Full details here. 
  • Teradata Aster is looking for Distributed Systems, Analytic Applications,  and Performance Architects. As a member of the Architecture Group you will help define the technical roadmap for the product.
  • The New York Times is seeking a developer focused on infrastructure to join its newsroom development team. Read the full description here and send resumes to chadas@nytimes.com.
  • New Relic is looking for a Java Scalability Engineer in Portland, OR. Ready to scale a web service with more incoming bits/second than Twitter?  http://newrelic.com/about/jobs
  • Aerospike is Hiring! You dream in C - and like it? Then join us as a Senior Distributed Systems Engineer or Client / Application Engineer. People covent your bag of tricks for troubleshooting systems and network issues? Join our Operations and QA team. See if these positions are a fit for you! 

Fun and Informative Events

Cool Products and Services

  • aiCache creates a better user experience by increasing the speed scale and stability of your web-site. Test aiCache acceleration for free. No sign-up required. http://aicache.com/deploy
  • New Benchmark shows Aerospike nearly 10x Faster than the Competition. Thumbtack Technology YCSB Benchmark shows Aerospike nearly 20x faster than Cassandra, Couchbase and Mongodb. Read it now!
  • ScaleOut Software. In-Memory Data Grids for the Enterprise. Download a Free Trial.
  • LogicMonitor - Hosted monitoring of your entire technology stack. Dashboards, trending graphs, alerting. Try it free and be up and running in just 15 minutes.
  • AppDynamics is the very first free product designed for troubleshooting Java performance while getting full visibility in production environments. Visit http://www.appdynamics.com/free.
  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.
  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

If any of these items interest you there's a full description of each sponsor below. Please click to read more...

03:45 Destabilizing the bullying power structure (3371 Bytes) » Seth's Blog

Bullies aren't welcome. For every bully, there are a dozen or a hundred workers/kids/individuals that would prefer not to be bullied. Given these overwhelming odds, how do bullies continue to get away with it?

Bullying is what happens when an individual with power exercises that power against people who don't fit in. By threatening to expose or harm or degrade the outlier, the bully reinforces the status quo in a way that increases his power. [Physical bullying is a different phenomenon... I'm mostly writing here about emotional bullying.]

"I will punish you because you don't fit in, and I will continue to punish you until you do."

Bullying persists when bureaucracies and hierarchies permit it to continue. It's easier to keep order in an environment where bullying can thrive (and vice versa), because the very things that permit a few to control the rest also permit bullies to do their work. The bully uses the organization's desire for conformity to his own ends.

At the fabulous lab school in Manhattan, they're making huge progress at undoing this problem. A recent assembly (organized and run by students and volunteers) was created around weirdness, fear and most of all, "owning it." (The adults in these videos were only 10% as honest and risk-taking as the kids that stood up on stage. The kids talked about physical and mental disabilities, lifestyle choices and the things that made them sing).

When students are given permission to be their best selves, they take it, just as you and I would like to. Because, it's true, we are all weird. When there isn't a race to fit in the most, bullying those that don't fit in loses much of its power.

This is incredibly brave and risky for those in charge. It involves trusting people to become something wonderful, as opposed to insisting that they fit in at all costs. 

We're all a lot weirder than we'd like the world to know. Given the chance, we can share that weirdness and run with it. It's our best shot at a world with art, and a world without bullies. (More here, but even better, go do this in your organization...)

Weird shirt

 

  2013-02-18 Mon

02:30 Planting, harvesting and your fair share (978 Bytes) » Seth's Blog

When there is scarcity, we worry a lot about getting our fair share—what goes to him doesn't go to me. The harvest becomes fraught with danger and competition.

When we worry more about planting, though, sharing the harvest gets a lot less complex.

Plant enough seeds and the scarcity eases. In fact, if you plant enough, you'll never have to think twice about the harvesting.

  2013-02-17 Sun

03:15 Excoriated (1147 Bytes) » Seth's Blog

There are only three reasons to really chew someone out for something they did, only three reasons to have an emotional tantrum, to use cutting language and generally make them feel lousy:

1. You want them to never do it again.

2. You want them to stop doing it right now.

3. You feel upset about the change and taking it out on the person who took action makes you feel better. First clue, "he deserved it!"

Can we agree that the third reason is selfish and there are almost certainly better responses if your goal is one or two?

  2013-02-16 Sat

03:30 Is a famous thinker better than a great one? (2867 Bytes) » Seth's Blog

Does a bestselling author have more to say than someone who has written a brilliant book that didn't sell?

Does a tenured professor at Yale deserve more credence than someone doing breakthrough work at a local state school?

If the violinist in the subway has played to packed houses, does that make him better than the previously unknown singer around the next corner?

For physical goods, a trusted brand name certainly increases the likelihood of purchase, because the risk is lower. We figure that Nabisco is less likely to sell us an unflavorful dust cookie than some unknown brand at the health food store. For a new flavor, the brand makes it an easier choice.

An idea is different, though, because the only apparent cost is the time it takes to hear it. (That's not really true, of course).

And yet we hesitate to invest the time to hear ideas from lesser-known sources. It's not fair to the unknown inventor, but it's true.

I think this is changing, and fast. The permeability of the web means that you don't have to start at the top, don't have to get picked by TED or a by a big blog or by anyone with influence. Pick yourself.

It's true that when you pick yourself, people aren't as likely to embrace your idea (at first). That's because the personal risk of hearing new ideas from new places is the fear that our opinion of the idea might not match everyone else's. The real risk of interacting with unproven ideas is the fear that we might not react in a way our peers expect. The desire to fit in often overwhelms our curiosity.

It takes quite a bit of work (and a lot of luck) to acquire a level of fame. The question that might be worth asking is whether or not that effort is related to the quality of ideas underneath. Harvard has been around for nearly 400 years. That doesn't mean the brand name is worth as much as we might be inclined to believe.

Branding started with pottery, beer and biscuits. Now it affects the way we think about ideas, people and even science. Buyer beware.

  2013-02-15 Fri

09:30 Stuff The Internet Says On Scalability For February 15, 2013 (1954 Bytes) » High Scalability

Hey, it's HighScalability time: 

  • The Herokulypse. A cautionary tale of what can happen when scalability is left for later. Rap Genius created quite a stir (reddit, Hacker News) when they documented high costs ($20K/month for 15 million monthly uniques) and poor performance (6 second average response times) using Heroku's random routing mesh. The cause was tracked to queuing at the dyno level when the expectation was requests are routed to free dynos. Heroku admits this is a problem. So poor load balancing combined with RoR single threading = poor performance, one that adding more dynos and spending more money won't necessarily help. While it seems clear Heroku didn't make this aspect of their system crystal clear, the incident has generated a lot of teaching moments, if you slog through it all. This is a developing story.
  • You need money to feed the beast. Fred Wilson has some revenue ideas for you: Paid App Downloads - ex. WhatsApp; In-app purchases - ex. Zynga Poker; In-app subscriptions - ex. NY Times app; Advertising - ex. Flurry, AdMob; Digital-to-physical - ex. Red Stamp, Postagram; Transactions - ex Hailo. 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...