Data Engineer

*Job Role* We are seeking a skilled and experienced Data Engineer to join our team. The ideal candidate will play a critical role in designing, developing, and maintaining our Claims Data Warehouse (CDW), which is built on Snowflake and fed by Azure-based data services. The role requires deep, hands-on expertise in Snowflake, strong working knowledge of the Azure data ecosystem, and extensive experience with medical claims data, including professional, institutional, and pharmacy claims, eligibility, and provider data. You will combine SQL, data modeling, and performance optimization skills with a solid understanding of how healthcare claims are adjudicated, adjusted, and reported, to ensure the efficiency, scalability, and reliability of our data infrastructure. *Responsibilities Include:* * Design, build, and optimize the Claims Data Warehouse in Snowflake, including databases, schemas, tables, views, stages, file formats, and workloads for loading, transforming, and sharing data. * Build continuous and batch ingestion into Snowflake using Snowpipe, COPY INTO, external stages, and Azure Blob Storage/ADLS Gen2 landing zones. * Implement incremental and near-real-time processing with Snowflake Streams, Tasks, and Dynamic Tables, including change data capture (CDC), schema drift handling, and error-recovery patterns to keep pipelines reliable. * Implement medallion architecture (Bronze, Silver, and Gold layers) in Snowflake, from raw claim file ingestion through standardized, deduplicated claim data to curated, analytics-ready marts. * Tune Snowflake performance and cost by right-sizing virtual warehouses, using Query Profile, clustering keys, search optimization, materialized views, and resource monitors, and by resolving latency, queuing, and storage issues. * Enforce Snowflake security and governance, including role-based access control (RBAC), dynamic data masking, row access policies, object tagging, and secure data sharing, to protect PHI and meet HIPAA requirements. * Use Snowflake Time Travel, Fail-safe, zero-copy cloning, and replication to support backup, recovery, testing, and business continuity/disaster recovery (BCDR) strategies. * Write efficient, well-documented SQL (and Snowpark or stored procedures where appropriate) to support extraction, transformation, and loading (ETL/ELT) of large claims data volumes. * Model claim header and claim line structures and manage the full claim lifecycle, including adjudication status, adjustments, reversals, voids, resubmissions, denials, and coordination of benefits (COB), so that the most current and accurate version of each claim is always available. * Apply and maintain healthcare code sets and reference data such as ICD-10-CM/PCS, CPT/HCPCS, revenue codes, DRG, NDC, place of service, type of bill, and modifiers, and map them accurately across source systems. * Develop claims-based data products to support utilization management, case management, cost and trend analysis, PMPM reporting, provider and network analytics, quality measures (e.g., HEDIS), risk adjustment, and financial reporting such as incurred-but-not-reported (IBNR) and claim run-out analysis. * Perform regular claims data quality and reconciliation checks, including financial balancing of billed, allowed, and paid amounts, duplicate claim detection, eligibility-to-claim matching, and completeness and timeliness validation against source systems. * Partner with claims, medical management, actuarial, finance, and analytics stakeholders to translate business and clinical requirements into well-modeled, trusted data. * Build, schedule, and monitor ETL/ELT pipelines using Azure Data Factory (ADF) to move claims and cases data from source systems, SFTP, and payer/vendor files into Snowflake, with parameterization, alerting, and retry logic. * Use Azure services such as ADLS Gen2/Blob Storage, Azure Key Vault, Microsoft Entra ID, Azure Monitor, and Azure SQL/SQL Server to build secure, observable, and cost-effective data platform components. * Maintain and migrate existing SQL Server Integration Services (SSIS) packages as needed, and help modernize them into ADF and Snowflake-based patterns. * Contribute to CI/CD, version control (Git, Azure DevOps), and infrastructure-as-code practices for Snowflake objects and Azure data pipelines. * Use AI tools such as Snowflake Cortex, GitHub Copilot, Microsoft Copilot in Fabric, and LLM-based assistants to speed up pipeline development, code generation, documentation, testing, and troubleshooting. * Optimize database performance by analyzing, monitoring, and fine-tuning SQL queries and database structures across Snowflake and SQL Server environments. * Develop and maintain data pipelines to ensure seamless data flow between various source systems and the EDW, and work with development teams to integrate data services with applications and business systems. * Monitor and maintain the health of the platform, addressing issues related to performance, latency, data freshness, and storage, and manage database security, backup, and recovery strategies to ensure data integrity and availability. * Keep up to date with emerging trends and best practices in Snowflake, cloud data engineering, healthcare data standards, and SQL optimization.