Job Overview

• We are seeking an experienced Data Architect (Databricks / Backend Data Engineer) to support a mission-critical U.S. Air Force program. The selected candidate will be responsible for designing, developing, and maintaining scalable data architectures and production-grade data pipelines within the ADVANA environment. This role focuses on building and optimizing Databricks-based data solutions using the Medallion Architecture (Bronze, Silver, Gold) to deliver reliable, high-quality data for enterprise analytics and reporting.
• The ideal candidate will have strong expertise in Databricks, Delta Lake, PySpark, SQL, Python, data pipeline development, and data validation, along with experience working in secure DoD environments.
• Key Responsibilities
• Data Architecture & Engineering
• Design, develop, and maintain scalable Databricks Delta Lake data architecture.
• Build and optimize Bronze, Silver, and Gold data pipelines using Medallion Architecture.
• Develop curated datasets for downstream analytics and reporting platforms.
• Design efficient Delta Table schemas and optimize storage and query performance.
• Implement business rules and transformation logic within the Databricks data layer.
• Data Pipeline Development
• Develop and maintain ETL/ELT pipelines using PySpark, SQL, and Databricks.
• Build automated ingestion pipelines from multiple authoritative data sources.
• Implement incremental loads, upserts, scheduling, monitoring, and automated workflows.
• Manage production deployments, pipeline monitoring, troubleshooting, and performance tuning.
• Optimize data processing for scalability, reliability, and cost efficiency.
• Data Quality & Validation
• Perform data validation, reconciliation, row count verification, checksum analysis, and performance benchmarking.
• Investigate and resolve discrepancies between source systems and transformed datasets.
• Maintain metadata, data lineage, schema documentation, and transformation documentation.
• Ensure data integrity, consistency, and compliance with organizational standards.
• Platform Administration
• Manage Databricks workspaces, clusters, job orchestration, and access control.
• Support platform operations, environment configuration, and production maintenance.
• Coordinate platform requests, issue tracking, and operational support activities.
• Monitor system performance and recommend optimization strategies.
• Collaboration
• Partner with data architects, visualization developers, and business stakeholders.
• Support integration with reporting and analytics platforms.
• Translate business requirements into scalable technical data solutions.
• Participate in technical reviews, architecture discussions, and continuous improvement initiatives.
• Required Qualifications
• Active U.S. Secret Security Clearance (or higher).
• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent professional experience).
• 5+ years of experience in Data Engineering, Data Architecture, or Backend Data Development.
• Strong hands-on experience with:
• Databricks
• Delta Lake / Delta Tables
• PySpark
• SQL
• Python
• Experience designing and implementing Medallion Architecture (Bronze, Silver, Gold).

Job Type: Contract

Pay: From $1.00 per hour

Education:

• Bachelor's (Required)

Experience:

• Database Architect: 6 years (Required)
• Medallion Architecture (Bronze, Silver, Gold).: 3 years (Required)

Security clearance:

• Secret (Required)

Work Location: Remote