Data Engineer

Job Title:

Data Engineer

Function:

Technical

Name of Job Holder:

 

Location:

BIPP, ISB, Mohali

Reports to position:

Project Manager

Band:

 

Reportees to Position:

None

 

 

Job Purpose

Design, build and manage the data pipelines and databases (both Relational as well as NoSQL) for a large national-scale spatio-temporal data warehousing, visualization, and analytics project.



Job  Outline

·         Develop the database architecture keeping in view the broader project requirements.

·         Design the database schemas and tables based on system requirements, keeping in view data storage, access, and management needs.

·         Use programming skills in Python or other open-source tools to build robust data pipelines.

·         Integrate data from a variety of sources, ensuring adherence to quality and accessibility standards.

·         Design and deploy database monitoring systems to ensure high availability.

·         Write and maintain documentation that includes database structures, data standards, procedures and definitions for the data dictionary (metadata).

·         Manage the security and disaster recovery aspects of the databases.

 

Job Specification

Knowledge / Education

Specific Skills

Desirable Experience/Skills

BE/BTech in
Computer Science/Computer Engineering/Information technology/Electronics
OR
Master of Computer Applications (MCA)

·2-5 years of progressively complex industry experience in the design and management of opensource database systems, namely Postgre SQL (preferred) or My SQL

·Hands-on programming experience in Python, R and/or Java along with BASH shell scripts.

·Comfortable working with Linux systems and experience with Linux utilities and commands.

·Demonstrated experience with sound software engineering practice, in particular software asset management – preferably using a Distributed Version Control System like Git or Mercurial.

 

·         Fresh graduates with demonstrable database skills may also be considered at the intern level.

·         Ability to leverage multiple tools and programming languages to analyze and manipulate data sets from disparate data sources.

·         Basic Linux administration knowhow

·         Knowledge of Big Data platforms like Hadoop and Spark.

·         Knowledge of No SQL databases like Mongo DB.

·         Knowledge of Amazon Elastic MapReduce (EMR) and Amazon Redshift

 

Job Interface/Relationships:

Internal

External

  • Dedicated project personnel
  • Other BIPP personnel
  • Other ISB teams and projects

  External partners

  Consultants

S.N

Key Responsibilities

% Time Spent

1

Design and implement a dynamic database architecture in consultation with broader project team and users. This will include both the hardware as well as the software aspects.

30

2

Designing the data provenance and metadata framework

15

3

Design and management of the database schemas and tables

25

4

Developing and maintaining scripts and tools for database monitoring and ensuring database uptime

25

5

Keeping abreast of latest technology and process developments

5

 

Total Time Spent on All Responsibilities

100%

 

SN

Key Result Area

Key Performance Indicator

Measure

Weightage

1

Establish Database Architecture

Development of database architecture as per the project requirements

Appropriate usage of latest hardware and software technologies

25%

2

Design Database Systems

Design of data provenance system and metadata framework

Design and management of database schemas and tables

Design, development and management of database systems

50%

3

Maintain Database Systems

Develop and maintain scripts and tools for database monitoring, and ensure database uptime

Keep abreast of latest technology and process developments

Maintenance, and smooth functioning of database systems with latest technologies incorporated

25%




Do you wish to apply? Share your profile

Contact us at

040 23187777

0172 4591800

Timings

Monday- Friday, 08:00 to 18:00