The emergence of streaming architectures—frameworks of software components built to ingest and process large amounts of data from multiple sources—is driving demand for better reliability and performance. Tech teams often code data to improve app performance by using so-called “message envelopes.” But these add complexity – and are often difficult and costly to debug.
Daniel Selans and Ustin Zarubin – engineers by trade who have worked at New Relic, InVision, DigitalOcean and Community.com – thought what was needed was a way to detect anomalous behavior in encrypted data streams. After experiencing issues with streaming data frameworks, they co-founded it stream valleywhich not only alerts users to streaming problems, but can transform in-flight data and reprocess broken data on-the-fly.
“We saw the need for more actionable insights for streaming data in distributed systems,” Selans told australiabusinessblog.com in an email interview. “Alternative approaches cannot introspect streaming data and instead rely on metadata-driven statistics. Furthermore, since most companies that use streaming also use some form of data encryption, there are no tools that can read that encrypted data.
In addition to monitoring for critical data issues, Streamdal uses AI, including natural language processing algorithms, to detect personally identifiable information in streams and take action (e.g. redact). The company also maintains an open source package, Plumber, that can be used to dig into data streams and connect disparate streaming systems.
Image Credits: stream valley
Potential future capabilities include providing more detailed lineage of data streams and analyzing data on the fly for schedule changes, Selans says.
Selans sees that Streamdal mainly competes with in-house technical teams who have put together purpose-built, tailor-made solutions for their employers. He was not at liberty to name many customers due to ‘contractual reasons’, but revealed that Recharge and ParkMobile are among Streamdal’s most well-known paying customers. Meanwhile, Plumber has been downloaded more than 150,000 times, claims Selans.
“We help enterprises monitor and semantically analyze billions of events across their event-driven architectures for data issues such as real-time schema changes that could otherwise lead to potential customer outages,” said Selans.
As for the current economic headwinds and whether they could affect business, Selans doesn’t believe that will be the case. “We believe that even with mass layoffs, companies should maintain their event-driven architecture that powers their distributed systems and may even require additional support to manage these complex systems,” he added.
Streamdal itself – a Y Combinator graduate – appears well positioned to weather the storm, having raised $5.4 million in a launch round led by Work-Bench with participation from Crosscut, Verissimo, Data Council and unnamed angel investors. To date, the company has raised $7.2 million in venture capital, which Selans says is being spent on strategic hiring (Streamdal has a ten-person team), product and go-to-market initiatives.
Kelley Mak, a partner at Work-Bench, added in an emailed statement, “With the proliferation of modern data architectures and the sheer volume of data processed in distributed systems, implementing the right data performance guardrails is for distributed systems a challenge for many. From financial services to highly regulated industries, it is critical for organizations to proactively respond to “bad data” to avoid customer-side failures. The founders have this pain in their lived past lives as engineers… we couldn’t agree more with their mission to be the data performance standard for event-driven systems for engineering teams.
