What are the responsibilities and job description for the Big Data Consultant position at ClifyX?
Job Title: Bigdata Engineer
Location: Chicago, IL 60601 (Onsite)
Employment Type: W2 Contract
Detailed Job Description:
We are seeking a skilled and experienced Big Data Platform Engineer who is having 7 yrs of experience with a strong background in both development and administration of big data ecosystems. The ideal candidate will be responsible for designing, building, maintaining, and optimizing scalable data platforms that support advanced analytics, machine learning, and real-time data processing.
Key Responsibilities:
Platform Engineering & Administration:
• Install, configure, and manage big data tools such as Hadoop, Spark, Kafka, Hive, HBase, and others.
• Monitor cluster performance, troubleshoot issues, and ensure high availability and reliability.
• Implement security policies, access controls, and data governance practices.
• Manage upgrades, patches, and capacity planning for big data infrastructure.
Development & Data Engineering:
• Design and develop scalable data pipelines using tools like Apache Spark, Flink, NiFi, or Airflow.
• Build ETL/ELT workflows to ingest, transform, and load data from various sources.
• Optimize data storage and retrieval for performance and cost-efficiency.
• Collaborate with data scientists and analysts to support model deployment and data exploration.