An Open Source Big Data Bus

The goal of DataFibers is
to simplify big data processing
and be your Data Heros

Simple, but Powerful

  • Various data connects and sinks
  • Stream and Batch trasnforms
  • Topics, Schema and Meta-data
  • Data enginering and science together

Complete Guideline

  • The Insignt of Big data processing
  • The value of Kappa architecture
  • Considerations of design
  • Best practice with DataFibers
  • Rest API Document

Services

  • Community Support
  • Professional Support
  • Consulting Service
  • Technical trainings
  • Meetup and Conference

Simple

Simply build, plug, and immediately subscribe your data anywhere at anytime.

FLEXIBLE

Batch, Stream, Real-Time, or Hybrid data processing are right at hand.

Powerful

Data landing, discovery, transfer, transform, cache, mining are all in one place.

Consulting

Explore the oppotunities from DataFibers and Big Data to business success

Support

We actively support development/deployment requests on DataFibers and queries on big data use cases.

Training

We have provided on-line and off-line big data professional trainings across world.

Know more about DataFibers?

Check out <<DataFibers Complete Guideline>>

Read Our EBook

From our blog

Here, we are sharing our experience and best practice of using DataFibers as well as other big data technology.

Read more

Harness Engineering: Deep Dive into Orchestration Logic with Harness CD

on April 22, 2026

In the realm of modern software delivery, orchestration is king. As deployments become more complex, involving microservices, multi-cloud environments, and intricate rollback strategies, simply pushing code is no longer sufficient. This is where Harness Engineering, specifically its Continuous Delivery (CD) module, shines. This deep-dive will move beyond surface-level introductions and explore the architectural patterns, practical challenges, and “under-the-hood” mechanics of how Harness CD empowers sophisticated deployment orchestration. Beyond the GUI: Understanding Harness CD’s Core Abstractions While Harness boasts a powerful UI, its true strength lies in the declarative definition of deployment strategies.

Continue reading

Read more

Demystifying Databricks: Under the Hood of Delta Lake, Photon, and Unity Catalog

on April 19, 2026

Databricks has become a cornerstone of modern data platforms, offering a unified approach to data engineering, machine learning, and analytics. While its intuitive notebooks and managed Spark clusters are widely appreciated, the true power of Databricks lies in its innovative underlying architecture. This deep dive will pull back the curtain on key components like Delta Lake, the Photon Engine, and Unity Catalog, revealing how they orchestrate to deliver performance, reliability, and governance.

Continue reading

Read more

Leading Cloud Tech. Stack Comparison

on July 1, 2023

Introduction In today’s digital era, businesses are increasingly adopting cloud computing to scale their operations, enhance flexibility, and reduce costs. Among the major cloud service providers, Amazon Web Services (AWS), Microsoft Azure, Google Cloud, and Oracle Cloud have emerged as dominant players in the market. Each offers a comprehensive cloud technology stack tailored to meet different business needs. In this blog, we’ll conduct a thorough comparison of these leading cloud technology stacks to help you make an informed decision when choosing the best-fit cloud provider for your organization.

Continue reading

Read more

Embracing Kubernetes, Goodbye Spring Cloud

on June 19, 2021

I believe many developers, after familiarizing themselves with microservices, realized that they thought they had successfully built a microservices architecture empired with Spring Cloud. But after the popular of kubernetes (K8S), they were curious and exciting of creating the cloud native microservices serivces. The Era of Spring Boot and Cloud In October 2012, Mike Youngstrom created a feature request in Spring Jira to support a containerless web application architecture in the Spring Framework.

Continue reading

Our Technologies