Daxko’s TechOps Team’s Top Tools for Monitoring

By | Culture, Industry, Life at Daxko, Technology | No Comments

Ask an engineer to “monitor” something and chances are you’ll get a response resembling something just short of the stages of love. It will start with an overwhelming excitement and sense of ease – you’ll hear something like “Oh, sure, that’s easy!” This will soon be followed by countless hours being poured in and an attachment to the project that defies logic as they run through countless iterations of things working, not working, being completely thrown out and rewritten only to start the cycle again. At the end, what started this whole thing will either be fully monitored, partially monitored, or you’ll have a heartbroken engineer who’s just given up. At Daxko we’ve been through this cycle, well, let’s just say a lot.

For those unfamiliar with what the Daxko TechOps team is, let me provide a quick synopsis. TechOps stands for Technical Operations, and at Daxko, it’s the team responsible for the health [performance, uptime], compliance, escalated support, and operation of our production systems.  Stop and think about that for a second. TechOps is responsible for ensuring that the Daxko products are 1) available, 2) not slow, 3) secure, and 4) that the data in them is consistent!  That’s quite a task, and our team is able to excel at it for 2 main reasons. The first is Daxko’s version of DevOps – the TechOps Team and the Development Teams are closely integrated and we work with the Development Teams to ensure they can utilize and consume the tools that we use or build. The second is that we’ve built, and continue to iterate on, a monitoring stack that allows us to keep up with what’s going on and troubleshoot complex issues. That stack is what I’m going delve into in this post.

Monitoring means lots of monitors.

The Consideration

When designing and building any type of solution, you have to stop and consider the purpose of what you’re building and any constraints that may be in place. For Daxko, we have products that run in multiple environments (both hosted and cloud-based), using multiple database engines, and using multiple different technology stacks. Therefore, our core solution needs to be OS agnostic, scale easily, and be very malleable. On top of that, we have product-specific items that must be monitored and developers that want to know when things break. Great problems to have, but also lots of added complexity!

The Stack

To say our stack is complete is like saying that Microsoft is done adding “features” to Windows. Our stack is always in a constant state of flux, and being okay with that is part of what makes things work. Needs change all the time, and the system should be able to cope with that. And, the team should be okay with that.

So, what is our current stack composed of?

#1: Chef: Wait! Chef? What? Yep, Chef is a critical part of our stack. Without it, we don’t have consistency in our configurations and dependencies. Without that, everything else falls apart.

#2: Sensu: The beauty of Sensu is that it works on a complete pub/sub model where clients are configured to subscribe to certain channels and the checks for the channels are published by the Sensu servers. This means if we need another server of type X then we just spin it up and its check data automagically starts flowing in. Simple, efficient, and extremely powerful (also why #1 is so critical!).

#3: Graphite + Carbonator + Grafana: I admit it. I’m a HUGE Graphite fan.  I grew up in the days of MRTG and RRD and used Cacti exclusively for many years.  Graphite makes capturing, storing, and analyzing LOTS of data points extremely (well mostly) easy.  It is an extremely scalable system and allows for the creation of some very complicated graphs that just seem to appear instantly.  Couple that with Carbonator for capturing deep metrics on Windows systems and Grafana for building dashboards and you have a hands down rock solid solution.

#4: ELK Stack: We generate billions of log entries on a monthly basis, and we rely on the ELK stack heavily for ingesting, tokenizing, searching, and alerting on those logs. Our Kibana interface is utilized by everyone from Customer Success to TechOps to the Developers themselves!

#5: Pingdom: No system is faultless, and having an outside perspective is always a good thing. Pingdom provides us with that third party view, and we consider its data to be the gold for our uptime statistics. After all, if your servers are up but nobody can use them – are you really “up”?

#6: NewRelic: Daxko’s application stacks range from .Net to Java to PHP. One of our goals is to have single interface for viewing, trending, and comparing those products. NewRelic allows us to do this and allows us to collaborate between teams easily.

#7: pagerduty: Do I really need to explain this one? 😉

#8: Daxko Monitoring Service: You won’t find this one available for download anytime soon as it’s an internally developed tool, but it’s also one of our most powerful because it fills in the gaps left by the others. The Daxko Monitoring Service started life as a simple enough tool that was used for identifying and alerting on invalid data states that would crop up in our databases due to bugs in the software. Over the years, and multiple iterations, it has grown into a much more capable tool that still heralds back to its origins. However, in its latest iteration, it is also being expanded in a way that will allow our developers (on any team) to build test cases and execute them on timed intervals. This may seem like a trivial task; however, when you consider that some products have multiple (and some hundreds) of databases and you need the ability to be able to scale these test across all of those in a quick and effortless way while also taking some business logic into account – a custom tool is the only way to go.

A lot of tools means a lot of interfaces and honestly, that’s just a plain pain. Compound that with lots of people wanting more visibility into what’s going on, and this becomes very overwhelming.  So how do you handle that? We searched for a tool to help us solve this problem and in the end settled on a plan that will extend our Monitoring Service UI to incorporate data and alerts from all of our external systems into a simple, easy to use, interface that will give our team members the ability to subscribe to events and alerts on a product by product / service by service level.  Our hope is this will help to increase the collaboration between teams by making it easy for team members all over Daxko to get visibility into what is happening behind the scenes, leading to faster fixes and happier customers.


Ed M. is Daxko’s Director of TechOps and Lead Infrastructure Architect who has a passion for problem solving, system architecture, and embedded electronics.

YMCA of Greater Dayton Uses Daxko Payment Services to Automatically Update Member Card Data

By | Customer Experience, Industry | No Comments

The YMCA of Greater Dayton was looking for a payment processor that could offer their association more features without sacrificing exceptional service to members. Specifically, the YMCA was seeking ways to reduce processing fees, reduce returns, eliminate avoidable declines and decrease labor costs.

Pamela Flatter, IT Training & Operations Support Director, and LeAnn Krintzline, Accounting Manager were pleased with their current processing, but wanted upgrades that addressed these items as well as offered a single source of information and service. They found that Daxko Payment Services not only met with their current needs but also satisfied their business objectives.

Serving Members Better
“We have 10 branches, a resident camp, over 40 program sites and we serve more than 40,000 membership units. The majority of our members pay by credit card, so we needed a payment processor that could help us serve those members better,” explains Flatter. The team was particularly intrigued by Daxko Payment Services’ Proactive Account Updater Service. Daxko’s Proactive Account Updater Service automatically checks for outdated cards prior to the scheduled draft dates and updates cards automatically. This service reduces the number of declines, reduces membership cancellations, and saves staff time managing credit card issues.

“Daxko Payment Services was the only processor we looked at that had the Account Updater functionality and we knew it would help us serve our members better,” says Flatter. “With Daxko Payment Services, our conversations can be more focused on connecting with members rather than the unpleasant task of hassling them for updated card information.”

Automatically Reducing Declines
The team at the YMCA of Greater Dayton is pleased with Daxko Payment Services. The first time that Proactive Account Updater Service was run, more than 4,000 member cards were updated in Daxko Operations. Since then, credit card returns are down by 20% on subsequent draft dates. “For us to see returns decrease this quickly in just one draft, made our investment worthwhile,” says Krintzline. “And, to not have fees on returns is big for us,” adds Flatter.

Transaction Visibility
The YMCA of Greater Dayton is also quick to point out the benefits of increased transaction visibility they get with Daxko Payment Services. “I like having more visibility because it helps us better serve our members when we can explain why a card declined.”

Seamless Implementation
When asked about implementation, they agreed: “Actually, our implementation was seamless,” says Krintzline. “We have not had one hitch.”

From our Customers: Partnership

By | Industry, Mission Delivery, Organizational Health, Videos | No Comments

We spoke with Daxko customers and asked them how they view their partnership with Daxko. Here are their stories:

“I’ve seen daxko grow both in the number products and tools that they bring to the YMCA but also as a business and an organization and the knowledge they bring to us has changed over the years and I have definitely appreciated the partnership. In my role in San Francisco, we work closely with the Daxko team to look at new products and to help to develop the solutions that work best for our organization.”

Rachel Pacheco, Association Senior Director of Business Services
YMCA of San Francisco

“I can tell that they have taken the time to get to know us as an organization, to know our culture and to know our brand and they bring that into what they do. I don’t feel like we have to conform to Daxko’s systems or operations but they have brought us a product that helps us serve our communities the way that we are serving them, without having to change how we work.”

Leeann Buttram, VP of Member Services
YMCA of Memphis & the Mid-South

“If you are looking for a company that will support you not only in your software needs but also in the questions you have about trends and membership and ways to take care of those members. They’re partners, it’s like having an extended family at the other end of your computer…so to speak.”

Joe Paslay, President & CEO
Cleveland County Family YMCA

“They are invested in the Y. They know us they want to be a part of us. They sponsor things, they will lead workshops for us. It goes beyond that where they are constantly seeking feedback, through the community aspect where you can share good, bad, pros, cons. And, it’s not just this empty place we go put our thoughts, it’s read, it’s responded to, it’s added to the roadmap. You can kind of see that they are not just asking, but they are hearing and then they are taking the steps to do the things that we need to make our work in life easier and better.”

Angela Wallace, VP of Marketing & Communications
YMCA of Memphis & the Mid-South

“It has been great for us to shift our focus on engagement. Oftentimes we think about our wellness staff focused on things such as cleaning equipment, or simply helping someone fixing their form or keeping the wellness center clean and we’ve really shifted our focus and said we want to be more intentional about engaging with our members and having conversations with our members.”

Jarrod Covington, VP, District Executive Director
YMCA of Memphis & the Mid-South

“I have learned to appreciate Daxko and appreciate the tools it brings to the table. The staff team has everything at their fingertips for them to be able to be successful.”

Joe Paslay, President & CEO
Cleveland County Family YMCA

Knox County YMCA Uses Daxko Facility Access To Offer 24 Hour Services

By | Customer Experience, Facilities, Industry | No Comments

As our world gets more complex, wellness facilities are working to offer services that fit everyone’s lifestyle. That explains the rise in 24-hour services by many health and wellness facilities, including YMCAs, JCCs, and other health clubs. In fact, a 2015 IHRSA article states that recent years have seen “continued growth of convenient, always open (e.g. 24 hour)” facilities. Knox County YMCA in Illinois saw 24-hour access as more than a growth opportunity; they saw it as a way to better serve their community.

“Offering a 24-hour facility was high on our list of what we wanted to offer our members and community,” explains Kelly Corbin, Associate Executive Director of the Knox County YMCA. “In our community, there are a number of businesses and organizations that work non-typical hours and we wanted to be able to serve everyone. The last two years we tested this by staying open until midnight during the week.  This was well utilized and we realized we needed to move forward.”

In order to provide that type of access for members, Knox County YMCA needed facility access controls when the branch was not staffed. When looking for providers, Corbin was attracted to Daxko Facility Access which allows integration with their existing Daxko membership software. “After a little research, we were pleased to find that Daxko had a solution for us that would work perfectly,” states Corbin.

The ease of implementing Daxko Facility Access has made the transition smooth for the team at Knox County YMCA. Corbin emphasized that integrating facility access with their membership software was a “huge advantage” for their organization. “It is very easy to turn access on and off. Members are able to use their existing membership key card (or barcode on the Daxko Mobile app) to scan and gain entrance. There isn’t a need for an additional key fob. You can easily see when members have scanned in or if they have had trouble gaining entrance. You are also given an app for the software and can open the locked door with your phone from wherever you are if needed!”

Corbin and her team wanted to offer additional facility access as an added benefit of membership. “One of the great benefits of a Knox County YMCA membership is that access to all of our branches is no extra cost, including the 24-hour facility,” she explains. Still, the increased services are benefitting both members and the Y. “It has been a great success. Year over year, we are up 1,000 members!”