Data Engineer

JOB PURPOSE

To understand data sets collected from different sources, to put them in a centralised data store ready for analytics and data science and to transform
them for business intelligence or machine learning. To apply software engineering principles to ETL development using a new set of tools. To create
and enhance products which benefit from the power of distributed processing. To understand the infrastructure and data architecture to identify reasons for successful and failed products.

RESPONSIBILITIES

 Data Architecture

• Consult and educate stakeholders on methods for streamlining and standardising data recording to ensure quality and accuracy.
• Develop, construct, test and maintain architectures, such as databases and large-scale processing systems to enable the usability, accessibility and
  transformation of data.
• Collaborate with other Data Architecture teams within Interswitch to identify synergies and drive a cohesive approach to data governance and architecture across the payment processing.

Performance Improvement through Business Intelligence

• Support creation of machine learning algorithms by applying standard statistical analysis or data preparation methods.
• Employ regression modelling to model the relationship between unique sets of data through statistically-proficient and engineering means. 
• Study consumption trends and segmentation analysis to inform marketing and business performance metrics.
• Narrow big datasets down to series of data that is presented to key business stakeholders.

Internal Communications and Capability Building

• Help others get the most out of internal communications systems by offering support and advice.
• Serve as an advocate for data-driven product design, evangelise insights on what is working and what is not to help drive incremental gains in pipeline and revenue.
• Act as mentor and coach to team members while fostering an environment of mutual respect and trust among senior-level team members.
• Maintain an understanding of relevant technology, external regulation and industry best practices through ongoing education, attending conferences and reading specialist media.
• Deepen understanding of underlying structures that constitute a data pipeline and manipulate these structures to ensure quality assurance and to enable data scientists to build effective algorithms, models and tools.
• Use statistical and engineering techniques to apply data processing tools to real world business structures.

Technical Database Support

• Design distribution of basic database resources and provide physical modeling and design services to tune database applications for optimum performance.
• Assist in the development of large scale data structures and pipelines to organise, collect and standardise data that helps generate insights and addresses reporting needs.
• Apply understanding of key business drivers to inform decision-making.
  Uses expertise, judgment and precedents to contribute to the resolution of moderately complex problems.
• Interface with cross-functional teams to define data requirements and source data, either in terms of raw data or modelled data and logic to support insights, dashboard, and report requirements that inform decision-making.

Advanced and Predictive Analytics

• Support interpretation of advanced and predictive analytics data, using specialised software tools and functionalities.
• Understand a myriad of technologies in-depth, pick the right tool for the job and write code in Scala, Java, or Python to create resilient and scalable solutions.
• Assess performance and recommend next steps as new products are launched and provide new measurement methodologies on key metrics to assess the success and failures of a product.

BEHAVIORAL COMPETENCIES

Manages Complexity

• Makes sense of complex, high quantity, and sometimes contradictory information to effectively solve problems. For example, looks at complex issues from multiple angles; explores issues to uncover underlying issues and root causes; sees the main consequences and implications of different options.

Plans and Aligns

• Plans and prioritises work to meet commitments aligned with organisational goals. For example, outlines clear plans that put actions in a logical sequence; conveys some time frames. Aligns own work with relevant workgroups. Takes some steps to reduce bottlenecks and speed up the work.

Tech Savvy

• Anticipates and adopts innovations in business-building digital and technology applications. For example, investigates technologies to learn some cutting-edge best practices. Uses digital / social media to benefit the team and add value to the work being done; understands how to avoid misuse of these tools.

Ensures Accountability

• Holds self and others accountable to meet commitments. For example, accepts responsibility for own work, both successes and failures. Handles fair share and does not make excuses for problems. Usually meets commitments to others.

Collaborates

• Builds partnerships and works collaboratively with others to meet shared objectives. For example, readily involves others to accomplish goals; stays in touch and shares information; discourages "us versus them" thinking; shows appreciation for others' ideas and input.

Communicates Effectively

• Develops and delivers multi-mode communications that convey a clear understanding of the unique needs of different audiences. For example, listens attentively and takes an interest. Keeps others well informed; conveys information clearly, concisely, and professionally when speaking or writing.

Cultivates Innovation

• Creates new and better ways for the organisation to be successful. For example, constantly looks for new ideas and innovative ways of doing things; makes the effort to get involved in new areas. Offers original ideas and promising new solutions.

Optimises Work Processes

• Knows the most effective and efficient processes to get things done, with a focus on continuous improvement. For example, uses metrics and benchmarks to monitor accuracy and quality. Takes steps to make methods productive and efficient. Promptly and effectively addresses process breakdowns.

Customer Focus

• Builds strong customer relationships and delivers customer-centric solutions. For example, keeps in contact with customers to ensure problems are resolved, or to improve customer service. Studies customer feedback and emerging customer needs and uses these to determine some creative new ideas.

Balances Stakeholders

• Anticipates and balances the needs of multiple stakeholders. For example, draws upon insights from varied sources to gain a rich understanding of how to meet the needs of multiple internal and external stakeholders. Probes deeply in order to gain a rich, detailed grasp of the priorities of different stakeholders; takes initiative to respond to stakeholder problems.

Business Insight

• Applies knowledge of business and the marketplace to advance the organisation's goals. For example, shows considerable business insight, beyond the fundamentals. Asks probing questions and draws on a variety of sources to gain insight and to explore business drivers or industry trends

SKILLS

Data Collection, Analysis and Control

• Works without supervision and provides technical guidance when required on analysing data trends for use in reports to help guide decision making.
• Works without supervision and provides technical guidance when required on acquiring, organising, protecting and processing data to fulfill business objectives.

Business Requirements Analysis

• Works without supervision and provides technical guidance when required on analysing the business requirements that IT solutions must meet.
• Determine acceptance and evaluation criteria; Prioritisation; Observation; Focus groups; Analysis Skills.

Policy, Procedures and Compliance Management

• Works without supervision and provides technical guidance when required on developing, monitoring, interpreting and understanding policies and procedures,
  while making sure they match organisational strategies and objectives.
• Works without supervision and provides technical guidance when required on achieving full compliance with applicable rules and regulations in management and operations.

Application Development

• Works without supervision and provides technical guidance when required on developing software using programming languages.

Verbal and Written Communication

• Uses clear and effective verbal and written communications skills without supervision and provides technical guidance when required on expressing ideas, requesting actions and formulating plans or policies.

Architecture

• Works without supervision and provides technical guidance when required on designing architectures that meet system and service requirements.

Presentation skills

• Needs guidance (but not supervision) to communicate with other people by speaking in a clear, concise and compelling manner

Planning and Organising

• Works without supervision and provides technical guidance when required on planning, organising, prioritising and overseeing activities to efficiently meet business objectives.

Review and Reporting

• Works without supervision and provides technical guidance when required on reviewing and creating relevant, lucid and effective reports.

Data Mining and Machine Learning

• Works with full competence to extract patterns from large amounts of data. Typically works without supervision and may provide technical guidance.

Customer Service Delivery

• Works without supervision while providing technical guidance as needed on meeting high customer service standards.

Project Management

• Works with guidance (but not constant supervision) to manage projects and Programmes within desired cost, time and quality parameters.
• Project Change Log / Register; Project Change Management Plan; Project Charter; Project Management Plan; Project Requirements Management Plan; Project Requirements Verification Plan; Project Resource Management Plan; Project Schedule; Project Schedule Baseline; Project Scope Statement.

Workflow Management

• Works with full competence to orient work in a workflow to plan, organise, and execute the steps it takes to achieve a goal to achieve higher efficiencies. Typically works without supervision and may provide technical guidance.
• Kanban Board (Workflow Management); Workflow Management.

Business Data Modeling

• Works with full competence to conduct activities to collect, analyse, diagram (model), and report information and data flow, including state changes, to help make strategic decisions, achieve major goals, and solve complex problems. Typically works without supervision and may provide technical guidance

Metadata Management

• Works with full competence to create and manage metadata systems and solutions to ensure information can be adequately found, shared, and maintained across an organisation. Typically works without supervision and may provide technical guidance.

Database User Interfaces and Queries

• Works with full competence to create and run queries and interact with various database interfaces and query languages.

IT Testing

• Works without supervision and provides technical guidance when required on ensuring that IT applications meet required specifications by designing, executing and reporting on tests of systems and services.

IT Data Management

• Works without supervision and provides technical guidance when required on managing information throughout its lifecycle by designing and executing appropriate architectures, policies and practices

IT Support

• Works without supervision and provides technical guidance when required on monitoring, diagnosing and fixing technological problems.

EDUCATION 

• University First Degree in Computer Science, Information Technology, Statistics, Mathematics, Finance or related fields.

EXPERIENCE 

• At least 6 years' relevant experience in Data Management roles ideally within financial or FinTech institutions, including a minimum of 3 years in data engineering.