Lead DevSecOps - Splunk

Senior Splunk Engineer | Location: Richmond, VA | Contract

We are seeking a highly skilled Senior Splunk Engineer to support enterprise-scale observability, logging, and infrastructure initiatives within a dynamic, hybrid environment. This role focuses on designing, implementing, and maintaining scalable and secure data pipelines and logging ecosystems across cloud and on-premises infrastructures, primarily leveraging Splunk and Cribl platforms. The ideal candidate will have a strong background in platform engineering, cloud architecture, and DevOps practices, contributing to critical infrastructure migration and optimization projects.

Key Responsibilities

• Designs, implements, and maintains distributed Splunk environments, including search heads, indexers, forwarders, and deployers, ensuring high availability and performance.
• Manages indexer clustering and search head clustering configurations to support scalable and resilient logging systems.
• Configures data inputs, parsing rules (props/transforms), and manages index lifecycle policies to optimize data ingestion and storage.
• Develops dashboards, alerts, and advanced queries to facilitate monitoring, analytics, and operational insights.
• Optimizes platform performance through capacity planning, troubleshooting ingestion and search issues, and tuning system components.
• Implements role-based access controls (RBAC), authentication integrations, and security measures to safeguard data and platform integrity.
• Designs and executes data onboarding strategies, including configuring HEC (HTTP Event Collector) and managing Splunk DB Connect integrations.
• Ensures data quality, consistency, and accessibility across the platform for accurate analytics and reporting.
• Deploys and manages Splunk infrastructure on AWS, including EC2 instances, VPCs, security groups, and storage solutions such as S3 SmartStore and EBS.
• Implements cloud monitoring and automation using CloudWatch, Auto Scaling, and Systems Manager to enhance operational efficiency.
• Builds and maintains infrastructure as code using Terraform or CloudFormation, and automates configurations with Ansible, Puppet, or Chef.
• Develops scripts in Python or Bash to streamline operational workflows and integrates automation into CI/CD pipelines.
• Deploys and manages Cribl Stream and Edge architectures, designing pipelines for data transformation, routing, filtering, and enrichment.
• Optimizes data ingestion costs through sampling, aggregation, and redaction strategies.
• Implements security and compliance controls, including encryption, IAM policies, and standards such as PCI-DSS, HIPAA, and SOC 2.
• Responds to and resolves platform incidents, including root cause analysis for P1/P2 issues, supporting on-call rotations.
• Collaborates with vendors and internal teams to ensure platform stability, security, and continuous improvement.
• Participates in Agile ceremonies, contributes to backlog refinement, and promotes DevOps best practices within the team.
  
  

Required Qualifications

• 6–8 years of experience in software engineering, DevOps, or platform engineering.
• 2 years of hands-on experience with Splunk architecture and administration.
• Proven experience deploying and managing Splunk in cloud environments, particularly AWS.
• Strong knowledge of container platforms such as Kubernetes, Docker, or OpenShift.
• Expertise in Infrastructure as Code using Terraform or CloudFormation.
• Proficiency with automation tools like Ansible, Puppet, or Chef.
• Experience with CI/CD pipelines and DevOps practices.
• Linux system administration skills.
• Ability to troubleshoot complex platform issues and optimize system performance.
• Excellent communication and collaboration skills.
  
  

Preferred Qualifications

• AWS Certification (Solutions Architect or Developer).
• Master’s degree in Information Technology or related field.
• Experience with Java full-stack development and cloud-native architectures.
• Familiarity with microservices, serverless architectures, and modern web frameworks (React, Angular).
• Knowledge of API integrations (REST, API Gateway, MuleSoft).
• Experience with monitoring tools such as ELK, CloudWatch, or Splunk.
• Prior experience working in financial services or regulated environments.
  
  

About Delphi-US - Peacemakers In The Talent War

Delphi-US is a national IT Services firm based in Newport, Rhode Island. Delphi attracts our nation's best and brightest Technical & Professional Talent and positions our nation's finest employers to further their competitive advantages. Delphi's core technical focus includes Data Sciences & Analytics, Modern AI/ML Model Monetization, Cloud Engineering, Cyber Security, ERP Rescue; serving domains from sensitive national energy sectors to critical financial services, biomed, lifesciences & pharma, major government & municipal institutions, and much of the USA's Fortune 500 corporations. Delphi Associates are deployed on critical IT Projects where their consultative business acumen, comprehensive technical expertise, and committed service mission drives our client's success. Our proprietary skill-based and cultural matching process results in higher qualified project teams, custom curated for our client’s specific technical & functional goals. We are supremely experienced, friendly, professional, and ready to advocate on your behalf. Delphi has an indelible understanding of employer expectations, a track record of project success and we deliver lasting results.

6064