Teradata has announced to develop Kylo, a data lake management software platform built using the latest open source capabilities such as Apache Hadoop, Apache Spark and Apache NiFi. The platform is expected to deliver unprecedented efficiencies for companies creating data lakes.
Kylo is a Teradata sponsored, open-source project that is offered under the Apache 2.0 license. Kylo evolved from code harvested from proven data lake engagements led by Think Big Analytics, a Teradata company, which will provide services and support for Kylo.
With substantive experience creating business value from data lakes, Teradata is contributing Kylo to help organizations address the most common challenges they face in data lake implementation efforts. These include the central problem that data lakes simply take too long to build, and in the average 6-12-month build cycle, users find that use cases can become out of date and less relevant to quickly evolving businesses. Second, despite the lower cost of software, engineering costs quickly mount. Finally, a data lake, once created, may fail to attract users who find it difficult to explore, and so little value is realized.
Derived and developed from data lake deployments across industries, Kylo can easily help resolve these challenges, because it integrates and simplifies pipeline development and common data management tasks, resulting in faster time to value, greater user adoption and developer productivity. With Kylo, no coding is required, and its intuitive user interface for self-service data ingest and wrangling helps accelerate the development process. Kylo also leverages reusable templates to increase productivity.
“Open source technology is making strong inroads into mainstream enterprise technology and has become an important consideration for enterprises looking to develop a modern end-to-end enterprise data architecture. In today’s highly competitive landscape, businesses are increasingly looking at driving more efficiency with an advanced data lake solution that is open and easy to integrate into other business-critical platforms without losing out on necessary controls especially in sensitive industries like health care, financial services, telecom and retail. Our new open source software platform will help companies to load data faster into their data lakes enabling their businesses to drive constant innovation using real-time data analysis.” said Sunil Jose, managing director, Teradata India.
“Many organizations find that implementing big data solutions on the Hadoop stack is a complex endeavor. Big data technologies are heavily oriented to software engineering, developers and system administrators,” said Nik Rouda, senior analyst with Enterprise Strategy Group (ESG). “Our research found 28 percent of organizations still struggle to staff teams with enough BI and analytics talent, much less big data and open source solution expertise. 77 percent of those surveyed say new big data initiatives will take between seven months and three years to show significant business value. It doesn’t have to be that way. I commend Teradata for open sourcing Kylo – an innovative and meaningful contribution.”
Encapsulating extensive experience from over 150 data lake engagements, Kylo helps organizations address the most common challenges they face in data lake implementation efforts, including; skill shortage for experienced software engineers and administrators; learning and implementing best practices around data lake governance; and driving data lake adoption beyond engineers.