The best way to prepare for the future is to understand the past: You have to know where you came from. And to direct future growth and change, you need a vision, and the right people and structures in place.
When DiscoverOrg started out in 2007, it was a data company. But at some point a few years out, as our database and features continued to grow – we realized that it was a technology company, too.
That’s a very hard change for a company to navigate. Many fail here.
This is our company’s story of how we outgrew our technology – and even some of our ideas – course corrected in the nick of time, and righted the ship. “It was like changing the tires on a racecar … as you’re driving around the track at 80 miles per hour,” says Andrew Steckley, Chief Data Scientist.
This technology transformation was instrumental in DiscoverOrg’s success today, and we’ve learned some unforgettable lessons moving forward.
Read on to see if anything sounds familiar.
Darrin Mossor, DiscoverOrg’s Software Development Manager, points to a few key areas we focused on, to grow in a way that produces an evolving, ever-better product for our customers:
- Build alignment with leadership.
- Build an effective, scalable organizational structure.
- Create culture that embraces communication and is committed to improving.
- Hire excellent folks, and never lower the bar. (We’re still hiring!)
- Create and monitor alignment with goals.
- Create an environment that advocates for best practices and rewards passion.
- Trust that your teams will work hard and try to do the right thing.
… And while it may not have been part of co-founder and CEO Henry Schuck’s original vision, DiscoverOrg managed to evolve from a company with an interesting product – to a technology powerhouse with a very interesting product.
[PRESS RELEASE] We just increased contacts by 400% and accounts by 200%! See how technology is driving our data strategy.
Outgrowing our old technology
Meet DOrg 1: The first platform.
DOrg 1 was built on an old shopping-cart application from the late ‘90s, running against a – single-point of failure – MySQL database.
In the frosty, early days of 2014, DiscoverOrg’s tech team was comprised of five internal technologists and a handful of contractors – all madly working to keep DOrg 1 up and running.
DiscoverOrg was built around the concept that data – and data accuracy – were all-important. Most resources were focused on building and cleansing the data itself. Much less attention was given to how the data was being delivered and maintained.
That focus on data quality distinguished DiscoverOrg and was key to success and growth in the early years.
But despite good strategy around data quality, a strong foothold in the market, and a visionary CEO, the biggest threat to the company’s success was its own technology platform.
It was time to become a tech company.
“We were growing, we were doing well, but there hadn’t been much attention to the technology that delivered the data, and how it was going to scale,” says Andrew Steckley. “I joined DiscoverOrg in 2014 to lead the technology team. It was clear pretty fast – the DOrg 1 platform was rapidly becoming a chokehold on the company’s growth. Within 3 weeks, I told Henry [Schuck]: We need to rebuild this thing from ground up. Right now. Mission critical.”
“DOrg 1 was overloaded even without any users on the system!” says Steckley. “It needed a massive redesign to distribute it into multiple applications and micro-services. That’s how to get a complex system to scale – it can’t be a monolith.”
Time for DOrg 2.
Transitioning to the cloud
In the cold, early months of 2016 – two years since our story started – DiscoverOrg’s Product Development team was deep in a rewrite of an entirely refactored version of the DiscoverOrg platform from scratch, based on the features available in the existing DOrg 1 product.
“Halfway through building out the DOrg 2 platform,” Steckley says, “I made the decision to move from a datacenter to a cloud infrastructure. “
The sooner we moved to the cloud, the sooner DOrg 2’s design could incorporate certain services available only in the cloud environment. Our plan was to replace every bit of software for our applications – AND switch the entire technology infrastructure to a new model.
That was really ambitious!
“Moving the platform to a cloud infrastructure was something we had to do mid-stride,” Steckley says. “We couldn’t do it in phases. Doing a roll-back, if something went wrong during the process, was not an option here. We had to do it right the very first time.”
They made the move one evening during the July 4th weekend.
It required over 100 specific steps within a short time window, and involved almost every person on the tech team. They completed it flawlessly.
“I don’t think many people realize how much activity went on that night down in the engine room,” Steckley says, “but that’s a good thing! That’s what success looks like, in this case.”
“As anyone who has had to do this before knows, rebuilding a platform is a big challenge.” adds Mossor. “This alone was an enormous accomplishment! Some succeed, but many fail. The small Product Development team that existed at that time earned us that success.”
The cloud environment offered a wealth of scalable architectures and services that simply weren’t available before.
DOrg 2 technology updates included:
- Java tech stack on a clustered database with cloud infrastructure
- AWS cluster database services (Aurora) on NOSQL “search database” in the cloud
- Tasks broken tech down into micro-services, scalable services
The move to the cloud made high growth possible for the company.
Quality, Depth, Quantity: See how technology is changing the way DiscoverOrg gathers and cleanses data.
Growing pains and communication breakdown
Of course, a technology issue is never JUST a technology issue.
Engineers and developers are good at building things, but they aren’t always good at knowing what should be built. That’s where Product Management comes in. The success of Product Development depends on a good relationship with Product Managers, who are able to effectively represent the various interests of different stakeholders.
And despite the success of the database migration, as we entered 2017, that relationship was rocky.
Poor communication between internal teams
Darrin Mossor explains: “We didn’t have the relationship we wanted with our partners in the Product Management team.” They were a small, scrappy team, driven to meet the demands of a diverse and very vocal group of stakeholders.
“This is a really challenging position for a Product team, and a Product Development team, because it can lead to a blurred vision and lack of clarity about goals. We were communicating well within our own organization, but we struggled to communicate and coordinate between different groups in the company.
“These are not uncommon challenges,” Mossor adds. “As a startup, DiscoverOrg had an ethos of ‘move fast and break things’ … which sounds great, until it’s your thing that’s moving too fast and breaking!”
By the time a feature request made its way to Development, there was a good chance it was not described well or was confusing, as it tried to address multiple requests and stakeholders.
The lack of clarity between groups, lack of precision, and misalignment, as we all tried to respond to different choruses of stakeholders, was a roadblock.
The hand-off from Stakeholder to Product to Development was troubled.
“Stay in your own lane”
“With several lanes of flight, sometimes affecting the same feature, we would inadvertently step on each other’s toes,” Mossor says.
“We were shooting for ‘agile,’ but we fell closer to ‘move quickly and figure it out as we go.’ This often resulted in missed expectations and disappointment from stakeholders. It was difficult to meet our commitments to timelines, quality, and product delivery.
“These were our darkest days. The relationship between Product Development and our Leadership was strained, despite a shared desire to accomplish the same goals.”
New leadership shines a light on strategy and structure
Growing pains needed fresh eyes, clear responsibilities – and a very clear, very strong chain of command.
Over the course of the next year, DiscoverOrg brought in Katie Bullard as the Chief Marketing Officer (now President), Justin Withers as Product Manager, Roger Cracel as Chief Technology Officer, Lead Architect Michael Helmeste, and Mark Johnson as Senior VP and Chief of Staff.
“The symptoms – taking longer to develop new capabilities that created frustration, low morale on the dev team; and an inability to communicate objectives – couldn’t be more clear,” says Mark Johnson.
“I created processes and cadences to elevate the conversation, and, most important: How do we get infrastructure on the roadmap?
“I implemented SLA (service-level agreements) – to help bridge communications gap. We put policies in place. (For example: Do we take ad-hoc data requests from sales? How quickly should we respond to requests?) We agreed on a charter and priorities for the development team.”
Another step Johnson took was to create a “fast lane.” “It takes too long to develop things, because short projects stack up behind the big ones. So we made a ‘fast lane’ – with dedicated resources.”
Things were no longer personal. Now it was about the process.
“When morale improves, you make better new hires. Engineering is excited again! You’re not waiting for the other shoe to fall. We started thinking more about development pathways for engineers, things that mature companies do, that are helpful to people as employees. We started exposing pathways to develop careers, so our people could take advantage of us as a growth company.
So how does a fresh, new strategic leadership team help solve a technology issue? Make space and stand back.
Getting technology on the roadmap
Johnson, Bullard, Withers, and other executives helped architect a product roadmap, which the product development team used to prioritize work.
“We had to sequester a certain amount of dedicated capacity for maintaining infrastructure,” Johnson says.
“The product infrastructure was built on anticipated needs from a couple of years ago. And adding new data and features took up all our capacity.”
With the new leadership, clear communication, and circumscribed expectations, the team was aligned – and ready to swap out the wheels on the racecar.
Healthier and well-organized, the tech team was able to focus on priorities – and deliver a superior product.
Just in time.
Stress-testing the tech stack
We had a foot on the gas, but DOrg 2 faced a series of challenges right out of the gate.
A lot of tech companies in the data space may find these familiar.
The first test: GDPR
This is a thorny issue for every single data company on the planet, who sells into the EU.
The first substantive demonstration of DiscoverOrg’s tech stack was the puzzle of GDPR (General Data Protection Regulation) – a new set of rules created by the European Union to give EU citizens more control over their personal data.
“We were able to deliver a great solution on an extremely short timeline,” says software developer Ryan Wheeler, “in an area where industry leaders like Google and Microsoft struggled.
“Everyone understood the plan. We maintained excellent communication between our own teams, as well as with Product and Leadership. So when unexpected obstacles appeared – and they did – we had a plan.”
Scaling our export service
The original design for our export service worked fine when we were a small company with less data and fewer customers.
But the current design was not going to scale. We quickly formed dedicated teams to create new infrastructure to take the load off the customer-facing application. The teams broke the work up into components, so the work could be completed in parallel.
Building an asynchronous library
The demand on DiscoverOrg infrastructure fluctuates. We could, of course, pay for a huge capacity that exceeds our anticipated demand. This method is costly and based on guesswork.
“The async library solves the problem differently,” explains Michael Helmeste, Lead Architect. “We decide the size of the workflow pipeline, and queue the jobs. This distributes the peak load over time; making it much more efficient, and less costly. More important, we have more control. If the queue grows larger than we want it to be, we increase the size of the pipe the work flows through.” It works great!
Future tech issues are inevitable, of course.
But with the proper technology in place – and the right talent at the helm – we’re in tip-top shape for a challenge.
“We are a data company. We are a go-to-market engine,” says Mark Johnson. “And in order to continue profitable growth, we’re becoming a tech company, too.”