Data Engineer

Website Federal Reserve Bank of San Francisco

The Federal Reserve Bank of San Francisco is looking for a Big Data Engineer to join the Advanced Data and Analytics Capabilities Team. We are a team based out of San Francisco that partners with business lines across the Federal Reserve System to deliver big data and advanced analytics products and solutions. In this role, you will have the opportunity to contribute to several high-quality data solutions and enhance your technical skills across many disciplines. We employ state of the art technologies that are part of the Hadoop ecosystem, which includes tools used for data integration, data modeling, and data analytics. You will have an opportunity to apply your critical thinking and technical skills across many disciplines.

In this role, you will contribute to high quality technology solutions that address business needs by developing solutions for the platform or applications for the customer business lines. You should have strong communication skills as you will work closely with other groups, including development and testing efforts of your assigned application components to ensure the successful delivery of the project.


  • Design, develop, and maintain end to end data solutions using open source, modern data lake, and enterprise data warehouse technologies (Hadoop, Spark, Cloud, etc.)
  • Contribute to multiple data solutions throughout their entire lifecycle (conception to launch)
  • Partner with business stakeholders to understand and meet their data requirements
  • Design, build, and maintain machine learning data pipelines
  • Maintain security in accordance with Bank security policies
  • Participate in an Agile development environment
  • Develop code in Big Data environments using Java/Python etc.
  • Lead and communicate to technical and business product managers, as well as third parties, on solution design
  • Act as a role model, thought leader, and change management for new software and technology throughout the company
  • Work on multiple projects as a technical team member or lead driving elaboration, design and development of software
  • Collaborate with Developers, DevOps, Release Management and Operations
    Maintain security in accordance with Bank security policies
    Participate in an Agile development environment by attending daily standups and sprint planning activities
  • Develop, execute, and document unit test plans and support application testing
  • Provide operational support for applications and utilities
    Tackle issues and participate in defect and incident root cause analyses
  • Assist in the deployment of new modules, upgrades, and fixes to the production environment
  • Independently determine methods and procedures on new assignments, and may provide work direction to others


  • Bachelors degree in Computer science, Information Systems, or other related field or relevant work experience
  • 5+ years data engineering and programming skills in Java and/or Python, including knowledge of Big Data ecosystem
  • Experience with the Hadoop ecosystem including HDFS data distribution, processing, workflow, Hive/Impala/Spark/Oozie
  • Experience programming and scripting on UNIX / Linux. (i.e. Python or Bash)
  • Experience with CTRL-M, Cron and scheduling of batch jobs
  • Experience performing operational support
  • Passion for technology and data, a critical thinker, problem solver and a self-starter
  • Strong quantitative and analytical skills
  • Ability to communicate effectively (both verbal and written) and work in a team environment
  • Ability to balance multiple assignments and shift gears when new priorities arise
  • Familiar with Agile methodologies
  • Must be a U.S Citizen or Green Card holder with intent to become U.S. Citizen

Nice to have:

  • Working experience at Government or quasi-Government organizations
    Cloud experience and using big data technologies on the Cloud
  • Professional experience optimizing machine learning workflows and maintaining data pipelines
  • Hands-on experience with a variety of big data (Hadoop / Cloudera, Cloud, etc.) and machine learning (Spark, AWS SageMaker, etc.) technologies

***Effective October 1, 2021, all employees must be fully vaccinated against COVID-19 or qualify for an accommodation from the Bank’s vaccination policy; the Bank will provide accommodations as required by law for individuals unable to be vaccinated due to medical condition or sincerely held religious belief.

We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment.

At the Federal Reserve Bank of San Francisco, we believe in the diversity of our people, ideas, and experiences and are committed to building an inclusive culture that is representative of the communities we serve. The Federal Reserve Bank of San Francisco is an Equal Opportunity Employer.

To apply for this job please visit