Senior Data Pipeline Engineer Palo Alto, CA Full-time no C2Cno relo The Database Engineer is a very important role within the Database Operations Team . This role collaborates closely with Game, Business Intelligence, Operations and entire team . As a Database Engineer you will be responsible for the production pipelines and helping to shape and scale it. You should love working with Java and have experience with Apache Storm and have worked with data pipelines and Hadoop. You will be viewed as an expert in your field and should have experience leading or mentoring a team. Responsibilities:Maintaining, improving and scaling production pipelines in Apache StormRe-designing and Migrating the pipeline from Apache Storm to Apache SparkMonitoring and resolving performance, and capacity issuesAssist Engineering teams with troubleshooting and fine-tuning jobsAutomation is in everything we do. Ability to create scripts and programs that would automate daily tasksWork closely with the Engineering teams in ensuring good practices are followedInvestigate and benchmark other solutionsRequirements:Minimum 5 years of developing on JavaExperience working with Apache Storm or Apache SparkHave had experience working with production pipelinesExperience in automation using pythonGood understanding and knowledge of Linux (CentOS)BS in Computer Science or a related fieldBonus:Knowledge/Experience in HadoopKnowledge/Experience in Elastic Search, Kibana and LogstashExperience with Graphite or other monitoring tools and implementing graphsHadoop, Elastic Search, Kibana, Logstash, Graphite, monitoring tools, java, Linux, Apache Storm or Apache Spark, python, production pipelines

Apply For This Job