Full Job Description
Position Details:
Title: Big Data Engineer
Location: Chicago, IL- Hybrid - 3x/week onsite
Compensation: $65-$68.26/hour
Interview Process: One Inperson round
Why open: New project
Scope of role: Experience with Kafka, Pyspark and real-time data processing, Elastic Search, Kibana (nice to have)
Project: Dealing with streaming data
Expected to accomplish:
- Processing Data in Real-time
- Building system from scratch
- Designing abstract layers
Must have:
- Strong communication skills
- Kafka
- Pyspark
- Real time data processing
- SQL
- Spark
- Coding/ development in Python
Nice to have:
- Elastic Search
- Kibana
- Designing system from scratch/ designing abstract layers
Required Skills:
- Strong SQL Skills - one or more of MySQL, HIVE, Impala, SPARK SQL
- Data ingestion experience from message queue, file share, REST API, relational database, etc. and experience with data formats like json, csv, xml
- Experience working with SPARK Structured steaming
- Experience working with Hadoop/Big Data and Distributed Systems
- Working experience with Spark, Sqoop, Kafka, MapReduce, NoSQL Database like HBase, SOLR, CDP or HDP, Cloudera or Hortonworks, Elastic Search, Kibana, etc.
- Hands on programming experience in at least one of Scala, Python, PHP, or Shell Scripting, to name a few
- Performance tuning experience with spark /MapReduce or SQL jobs
- Experience and proficiency with Linux operating system is a must
- Experience in end-to-end design and build process of Near-Real Time and Batch Data Pipelines
- Experience working in Agile development process and deep understanding of various phases of the Software Development Life Cycle
- Experience using Source Code and Version Control systems like SVN, Git, Bit Bucket etc.
- Experience working with Jenkins and Jar management
- Self-starter who works with minimal supervision and the ability to work in a team of diverse skill sets
- Ability to comprehend customer requests and provide the correct solution
- Strong analytical mind to help take on complicated problems
- Desire to resolve issues and dive into potential issues
- Ability to adapt and continue to learn new technologies is important
Job Information
Job Category:
Other
Spotlight
Employer
Related jobs
Registered Respiratory Therapist (Draft)
BJC Healthcare
Additional Information About the RoleJoin the Respiratory Team at BJC Healthcare's flagship hospital and academic campus. Barnes Jewish Hospital is dedicated to your growth and development. ...
Oct 2, 2024
St. Louis, MO
Physical Therapist - Belleville
BJC Healthcare
Additional Information About the RoleGREAT OPPORTUNITYMemorial Hospital in Belleville is seeking a Physical Therapist to work in our outpatient setting. Our state-of-the-art facility offers a di...
Oct 2, 2024
Belleville, IL
Retail Pharmacist - Float
BJC Healthcare
Additional Information About the RoleWe are seeking a collaborative and adaptable pharmacist to join our Retail Pharmacy team. In this position, you will float between BJC retail pharmacies...
Oct 2, 2024
St. Louis, MO