An Open Source Big Data Bus

The goal of DataFibers is
to simplify big data processing
and be your Data Heros

Simple, but Powerful

  • Various data connects and sinks
  • Stream and Batch trasnforms
  • Topics, Schema and Meta-data
  • Data enginering and science together

Complete Guideline

  • The Insignt of Big data processing
  • The value of Kappa architecture
  • Considerations of design
  • Best practice with DataFibers
  • Rest API Document

Services

  • Community Support
  • Professional Support
  • Consulting Service
  • Technical trainings
  • Meetup and Conference

Simple

Simply build, plug, and immediately subscribe your data anywhere at anytime.

FLEXIBLE

Batch, Stream, Real-Time, or Hybrid data processing are right at hand.

Powerful

Data landing, discovery, transfer, transform, cache, mining are all in one place.

Consulting

Explore the oppotunities from DataFibers and Big Data to business success

Support

We actively support development/deployment requests on DataFibers and queries on big data use cases.

Training

We have provided on-line and off-line big data professional trainings across world.

Know more about DataFibers?

Check out <<DataFibers Complete Guideline>>

Read Our EBook

From our blog

Here, we are sharing our experience and best practice of using DataFibers as well as other big data technology.

Read more

Leading Cloud Tech. Stack Comparison

on July 1, 2023

Introduction In today’s digital era, businesses are increasingly adopting cloud computing to scale their operations, enhance flexibility, and reduce costs. Among the major cloud service providers, Amazon Web Services (AWS), Microsoft Azure, Google Cloud, and Oracle Cloud have emerged as dominant players in the market. Each offers a comprehensive cloud technology stack tailored to meet different business needs. In this blog, we’ll conduct a thorough comparison of these leading cloud technology stacks to help you make an informed decision when choosing the best-fit cloud provider for your organization.

Continue reading

Read more

Embracing Kubernetes, Goodbye Spring Cloud

on June 19, 2021

I believe many developers, after familiarizing themselves with microservices, realized that they thought they had successfully built a microservices architecture empired with Spring Cloud. But after the popular of kubernetes (K8S), they were curious and exciting of creating the cloud native microservices serivces. The Era of Spring Boot and Cloud In October 2012, Mike Youngstrom created a feature request in Spring Jira to support a containerless web application architecture in the Spring Framework.

Continue reading

Read more

Spark SQL in Depth

on April 28, 2021

In this article, we’ll look at how Spark SQL working on data queries in depth. Checking Execution Plan Data Preparing create database if not exists test; create table if not exists test.t_name (name string); insert into test.t_name values ('test1'),('test2'),('test3'); Test Code Preparing Below Scala code is used with testing with blocking at the standard input at the end. In this case, we can see more details from Spark WebUI.

Continue reading

Read more

Apache Spark 3.1.1 Released :)

on March 10, 2021

Apache Spark 3.1.1 is released on March 2, 2021. It is milestone release for Spark in 2021. This version of spark keeps making it more efficient and stable. Below are highlighted new features and changes. Python usability ANSI SQL compliance Query optimization enhancements Shuffle hash join improvements History Server support of structured streaming Project Zen Project Zen was initiated in this release to improve PySpark’s usability in these three ways:

Continue reading

Our Technologies