Virtue Tech an AWS Data Engineering company is seeking an exceptional Big data engineer to join the team. This person will play a key role in providing the end-to-end data engineering solutions to support key business initiatives. The Data Engineer is responsible for developing data pipeline and data engineering components to support strategic initiatives and ongoing business processes for customers. This role includes migration to AWS using various AWS components.
- Focus on scalability, performance, service robustness, and cost trade-offs
- Continuous drive to explore, improve, enhance, automate, and optimize systems and tools to best meet evolving business and market needs
- Attention to detail, coupled with ability to think abstractly
- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Create prototypes and proof-of-concepts for iterative development
- Keen to learn new technologies and apply the knowledge in production systems
- Take complete ownership of projects and their development cycle
- Bachelors/Masters in Engineering, Computer science
- 7+ years of experience in Data engineering
- Strong communication skills
- Experience with big data tools like Hadoop, Spark, Kafka, fink, Hive, Sqoop etc.
- Experience with relational SQL and NoSQL databases like Mysql, Postgres, Mongodb and Cassandra.
- Experience with data pipeline tools like Airflow, etc.
- Experience with AWS cloud services like: EC2, S3, EMR, RDS, Redshift, BigQuery
- Experience with stream-processing systems like: Storm, Spark-Streaming, Flink etc.
Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.