This role will be crucial in building a scalable, reliable, and cost-efficient system for handling vast amounts of market trading data, real-time news feeds and a variety of internal and external data sources. […]
Responsibilities Design, build, and optimize scalable pipelines for ingesting, transforming, and integrating large-volume datasets (market data, news feeds and various unstructured data sources). Ensure data quality, consistency, and real-time monitoring using tools like DBT, 3rd party libraries that can facilitate data validation processes. […]
Apply advanced data management practices...