- Manage and maintain our Big Data infrastructure, including HBase, Elastic search, and Cloudera
- Perform configuration, tuning, and ongoing optimization of big data infrastructure and applications
- Develop and implement processes for data ingestion, processing, and storage
- Work with other members of the DevOps team to troubleshoot and resolve issues related to Big Data systems
- Work with data scientists and architects to understand their infrastructure and analytics needs and provide solutions
- Collaborate with Data Engineers and Data Scientists to ensure data is available and accessible for analysis
- Develop and maintain documentation related to Big Data systems and processes
- Stay up to date with advances in big data technologies and tools. Research and recommend new solutions as needed
- Educate and train junior DevOps team members on Big Data technologies and best practices
Requirements
- 0-3 Years of experience at the relevant field
- Good command of the English language
- Excellent analytical skills
- Excellent interpersonal skills
- Excellent written and verbal communication skills
- Ability to prioritize and organize multiple assignments/tasks
- Strong problem-solving mindset
- Ability to adapt to a constantly changing environment
Technical Skills
- Expertise in Hadoop ecosystem technologies such as HDFS, Hive, HBase, Spark, Kafka, Flink, and NiFi
- Experience with Hadoop-based Ecosystems like Cloudera including its different compartments like CDP, CDF, and CML
- Expertise in ELK Stack (Elastic Search, Kibana, LogStash) and administration of logging systems over them
- Knowledge of distributed computing, cloud computing architectures, and data processing frameworks, such as Apache Spark
- Experience with data ingestion and processing tools, such as Kafka and Flume
- Strong communication and collaboration skills
- Ability to train and educate junior team members on Big Data technologies and best practices
- Ability to work independently and as part of a team
- Expertise in Teradata, PowerBI Server, and Informatica is a plus
Education
B.Sc. in Engineering or Computer Science or relevant
