Versions Compared

Old Version 7

changes.mady.by.user George Zaronikas

Saved on

compared with

New Version 8

changes.mady.by.user Wojciech Trocki

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Associated JBoss community project(s):


Idea Proposals

AeroGear

...

NodeJS based Data Synchronization engine on top of Apache Kafka, Debezium and GraphQL

...

Subscriptions 

Summary of the idea:

GraphQL Subscriptions allow developers to build reactive data-driven applications where each client actively subscribes to new data and receive it immediately after is available.
However in traditional messaging systems when a client subscribes for the first time it is not 

not going to get any previous messages. Purpose of this work this  will be to provide offline enabled data synchronization platform on top of GraphQL, Debezium and Apache Kafka 

Project references

Candidates are going to work closely with the JBoss community by contributing to the following projects:


Required knowledge

  • Node.js (Understanding performance implications of event loop etc.),
  • TypeScript 
  • Basics of GraphQL, React
  • Apache Kafka

...

  • (Basic concepts, Consuming messages, Partitioning) 
  • Knowledge of one of the relational and NoSQL databases (Postgress, MongoDB)
  • Familiarity with the event streaming principles
  • Engagement with AeroGear and Debezium communities is essential!


Definition of the problem

When building microservices architecture various parts of the system can interact with the source database. Using publishing events from the server application is not always possible and it might require modifications in various servers that are often legacy.

When utilizing debezium and kafka we can connect directly to the database to get notified about any change that happened in the database instantly.

Debezium will be only involved in collecting and publishing the data to kafka. The purpose of this work is to build an opinionated Node.js layer that will consume Kafka messages to apply some specific filtering and publish messages back to clients using GraphQL subscriptions.


Possible goals:

  • Introduce a stateful live query layer for Graphback based on kafka stream (https://www.youtube.com/watch?v=g-asVW9JFPw)
  • Check the performance of the solution and identify bottlenecks for scalability. 
  • Build a suite of the integration tests for various Graphback use cases
  • Provide example app for Debezium
  • Build a Node.js OpenShift template application for Graphback that can connect with Kafka
  • Build out of the box subscription connector for Offix that will update underlying cache/db


Technologies

...

  • AeroGear DataSync
  • Debezium

Knowledge prerequisite: Nodejs, GraphQL, AeroGear DataSync, React, Apache Kafka

Github Repo: https://github.com/aerogear/datasync

Skill level: Beginner/ Intermediate 

Contacts / potential mentors: wtrocki@redhat.com 

...