- Build the next generation of ScoreFast Data pipeline architecture
- Work on specific customer data analytics project and perform data analytics.
- Build plugins crawlers to extract data from external sources
- Responsible for data pipeline systems operation.
- Prototyping new technologies for integration.
- Requires a B.S. degree or higher in the area of data science, computer science, statistics, mathematics, physics, engineering, operations research from a reputed university.
- 3+ years experience building maintainable large-scale data pipeline architecture
- Working experience in Spark both batch and real-time mode. Experience with other real-time streaming platforms such as Kafka and Storm is a plus.
- Experience in building ETL application for big data iusing Hadoop in cloud (AWS)
- Knowledge of Hive, HBase, Zookeper, Oozie is a plus
- Knowledge of Github, JIRA, Confluence, jenkins, Docker/Kubernetes is a plus
- Languages: Python, Java, Golang
January 16, 2018