Big Data Engineer
Crisp is a social media safety and crisis monitoring firm, and our technology is used by some of the world’s largest, best known brands, to provide fast detection of critical issues and crises. We help companies by spotting fake news and rumours being spread about brands on social media, which can damage sales and impact share price.
We also protect users of social media from harmful and unwanted user generated content, utilising hundreds of analysts and moderators operating 24/7 and in over 50 languages.
As a Big Data Engineer, you will be responsible for managing data projects, ensuring efficient flow, storage and management of our big databases, which are used for operational and analytical purposes.
Collaborating with other members of the Data Science team, you will maintain the Company’s BigQuery Database and manage the integration with the company’s other data sources (including ElasticSearch, Firestore, Neo4J and Salesforce). You will monitor and manage costs of the database, continuously seeking opportunities for optimising the data and realising efficiencies.
Directly Responsible Individual:
- Manage BQ data pipelines, flows and storage for efficient processing of operational and analytical data flows
- Manage API connections of data across departmental systems
- Manage acquisition of data from internal departmental systems for analysis
- Manage integration of data from multiple sources, including ElasticSearch, Firestore, Neo4J and Salesforce
- Support development of sound data schemas across the department for effective data management (ie would be good to have this person lead on the work to ensure we get a solid schema structure)
- Work self-sufficiently and actively seek out help/knowledge when faced with uncertainty
- Own key projects and ensure work is delivered within timescales and to a high standard.
- Carry out analysis in a transparent, repeatable and robust way.
- Communicate effectively with both the team and other key stakeholders on progress and results of projects.
- Actively engage in transformation as the organisation evolves the services it provides to customers
- Manage the company’s secondary database (Google’s BigQuery Database)
- Adjust the flow of the production data to and from the BigQuery Database (and table structure where required) in order to ensure that the necessary data needed for analysis and outputs is stored in an optimal way.
- Collaborate with our engineering department to ensure all necessary configuration information can be accessed through the Company’s API and is stored in the BigQuery database.
- Collaborate with our Site Reliability Engineering Team to manage the ongoing operation and maintenance of the BQ data.
To be successful in this role, you will need:
- Critical and analytical thinking abilities with great attention to detail
- Familiarity with or willingness to learn different analytical programming languages.
- The ability to adapt to changes in the team
- The drive and initiative to take on more responsibilities
- Strong ability to scope, organise and complete projects in a timely manner
- Excellent communication and interpersonal skills and have initiative and confidence to use own judgement
- Ability to analyse and interpret data with clarity to provide value-added insight to CRISP business.
- Excellent ability to build relationships across all levels
- Willingness to learn new skills and knowledge
- Demonstrable and commercial experience in a data analytics field
- BSc in related subject or (equivalent)
- Experience working with Google Cloud Platform
- Ability to use REST API
- Ability to manipulate data or experience in SQL
- Ability to communicate results and technical concepts to a non-technical audience
- Experience of working in teams and collaborating with others to deliver joint projects
- Familiarity with different types of databases such as relational databases, NoSQL databases and Graph databases.
- A good knowledge of SQL and Google BigQuery
- Familiarity with the Neo4J’s graph query language, Cypher
- Experience of working with big datasets
- Strong Excel skills and VBA knowledge a plus
- Ability to manage multiple projects under different stakeholders and competing timescales
- Proven ability to work and contribute effectively within an organisational setting
Our rewards are as unique as our culture, and we want to attract the best people and retain them.
Not only will we ensure that your development is key, but you will be joining a fantastic team of like-minded people who work together as one team to achieve a shared vision.We offer an excellent salary and benefits package which includes:
- Market competitive pay rates based your skills and experience
- Discretionary bonus scheme / commission scheme with payment based on revenue generated as a result of generated sales leads
- 33 days holiday including Bank Holidays (20 days in the US)
- Critical Illness insurance
- Life Insurance Cover
- Healthcare Cash Plan / Healthcare, dental and vision plan
- An attractive pension / 401k retirement plan scheme
- Cycle to Work Scheme
- Employee perks schemes offering discounts, rewards, giveaways and more
- Subsidised gym membership
- Mental health wellbeing portal and access to an in-house clinical psychologist
- Support and provision of supplies to facilitate home working
- Flexible working opportunities
'This work meets the requirements in respect of exempted questions under the Rehabilitation of Offenders Act 1974, any applicants who are offered work for this organisation will be subject to an enhanced check from the Disclosure and Barring Service (DBS). This will include details of cautions, reprimands or final warnings as well as convictions. A criminal record will not automatically bar a person from successfully taking up this post.