Job Description:

At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. We do this by driving Responsible Growth and delivering for our clients, teammates, communities and shareholders every day.

Being a Great Place to Work is core to how we drive Responsible Growth. This includes our commitment to being an inclusive workplace, attracting and developing exceptional talent, supporting our teammates’ physical, emotional, and financial wellness, recognizing and rewarding performance, and how we make an impact in the communities we serve.

Bank of America is committed to an in-office culture with specific requirements for office-based attendance and which allows for an appropriate level of flexibility for our teammates and businesses based on role-specific considerations.

At Bank of America, you can build a successful career with opportunities to learn, grow, and make an impact. Join us!

Role Overview:

We are seeking a Principal Splunk Engineer to lead the design, operation, and evolution of our large-scale Splunk Enterprise / Splunk Cloud deployment. The platform ingests multi-terabyte daily data volumes across security, infrastructure, and application domains and is a critical component of our SOC and threat-detection capabilities. The ideal candidate has deep expertise in Splunk architecture, large-scale data onboarding, performance optimization, SmartStore/Indexer Clustering, and security-focused use cases.

Key Responsibilities:

Platform Architecture & Operations:

• Architect, operate, and optimize a distributed, large-scale Splunk environment (indexer clusters, search head clusters, cluster masters, deployment servers, IDM, ADFS/SAML integrations)
• Lead capacity planning, index design, data retention strategies, and SmartStore lifecycle management
• Maintain high availability, scaling, and resilience across multi-site deployments (including DR strategy)
• Drive Splunk version upgrades, app updates, cluster maintenance, and platform hardening

Security Logging & SOC Enablement:

• Collaborate with SOC, Incident Response, and Threat Hunting teams to ensure high-quality security log ingestion
• Onboard and normalize logs from firewalls, EDR, identity platforms, cloud providers, network telemetry, and custom applications
• Develop and optimize detection content: correlation searches, risk-based alerting, data models, macros, lookups, summaries
• Ensure compliance with logging standards (MITRE ATT&CK mapping, CIS/SOC2/ISO27001 logging requirements)

Data Engineering & Observability:

• Build and manage ingestion pipelines, parsing, field extractions, CIM compliance, HEC configurations, and forwarder architecture
• Implement data lifecycle tiers, filtering strategies, routing, and ingestion controls to reduce cost and improve efficiency
• Optimize search performance, knowledge objects, summary indexing, and acceleration strategies

Governance & Best Practices:

• Establish Splunk development standards, dashboards, and naming conventions
• Mentor junior engineers and act as a technical escalation point for the team
• Maintain documentation, operational runbooks, and logging onboarding guidelines
• Partner with Engineering, Cloud, SecOps, and App teams to drive company-wide observability maturity

Required Qualifications:

5+ years experience administering large Splunk Enterprise or Splunk Cloud environments

Strong hands-on knowledge of:

• Indexer clustering, search head clustering
• SmartStore / S3-compatible object store design
• Universal/heavy forwarder architecture
• Ingest actions, parsing, props/transforms
• KVStore, RBAC, SAML, encryption

Deep experience with security log ingestion and SIEM use cases

Strong SPL expertise, including:

• Search optimization
• Summary indexing / data model acceleration
• CIM mapping and field normalization

Experience with Linux systems engineering, scripting (Python/Bash), and automation frameworks (Ansible, Terraform, GitOps preferred)

Preferred Qualifications:

Splunk certifications (Core Consultant, Enterprise Admin, Enterprise Architect, ES Analyst/ES Admin, or equivalent)

Experience with:

• Enterprise Security (ES)
• SOAR (Phantom or comparable)
• AWS/Azure/GCP cloud logging architectures

Familiarity with high-throughput message brokers (Kafka/FluentD/Cribl)

Background in cybersecurity engineering or threat detection

Skills:

• Automation
• Influence
• Result Orientation
• Stakeholder Management
• Technical Strategy Development
• Application Development
• Architecture
• Business Acumen
• Risk Management
• Solution Design
• Agile Practices
• Analytical Thinking
• Collaboration
• Data Management
• Solution Delivery Process

Shift:

Hours Per Week: