Most Powerful Open Source ERP

Anomaly detection in logs

Nexedi is looking for a python developer interested in implementing anomaly detection in logs to detect abnormal usage of online platforms. The goal of this effort is to better detect simple attacks on a public cloud platform.
  • Last Update:2020-05-22
  • Version:001
  • Language:en

Details

  • Task:Anomaly detection in logs ("AI for threat detection")
  • Preferred location:Lille (France)
  • Other locations:Paris (France), Munich (Germany)
  • Type:Internship or Job or PhD
  • Function:Developer
  • Duration:3-6 Months/Permanent
  • Reference:Offer-2020-Anomaly

Description

Nexedi is looking for a python developer interested in implementing anomaly detection in logs and detection of abnormal usage of online services.

Nexedi has deployed various online platforms used for private or public clouds. A competitive analysis shows that conventional public clouds may include threat detection features that probably do not exist yet in open source cloud platforms.

We are thus considering to introduce in SlapOS a new kind of promise (see "Promise Theory - Principles and Applications") dedicated to the detection of abnormal behaviour of running processes.

In other words, we would like to implement "AI for threat detection". Some early prototypes based on unsupervised machine learning have shown promising results to detect abnormal usage of an ERP. We expect however that a fully functional system will combine explicit rules (80%), supervised machine learning (15%) and a combination of unsupervised machine learning and human operation for the rest (5%).

The first step in this task will consist of defining what we want to protect ourselves against : processes deployed by SlapOS that are not used in a way they supposed to be used. For example, a database that is supposed to process a few transactions per second with a few users is not supposed to receive connection requests from thousands of different users. A database that is supposed to be used locally is not supposed to generate a lot of network traffic.

The second step in this task will consist of defining a kind of "operation envelope" (similar to "flight envelope" in aerospace) that processes should stay within. In case they leave the operation envelope, a monitoring alarm is triggered.

The third step could consist in using unsupervised machine learning to detect anomalies. Each time an anomaly is detected, a human studies the anomaly and classifies it. Supervised machine learning can then be used to reduce the number of anomalies processed by humans.

Opportunities

  • Master Python and Cython for log analysis
  • Master Wendelin big data platform for log analysis
  • Learn scikit-learn and other ML libraries
  • Learn SlapOS open-source edge cloud

Responsibilities

  • Understand the pros and cons of anomaly detection for threat detection
  • Understand the pros and cons of operation envelope for threat detection
  • Contribute to SlapOS promises for anomaly detection
  • Contribute to open source projects such as Cython, NumPy, scikit-learn, SlapOS, NEO, etc.
  • Contribute to research projects to build the future of our open source stack

Requirements

  • Passionate, self-driven.
  • Willingness to contribute to an open source ecosystem and the Free Software community.
  • Good skills in GNU/Linux operating system.
  • Very good programming skills in Python
  • Very good software development skills (version control, testing, debugging).
  • Good command of English.

References

About Nexedi

Nexedi has been developing free software since launching in 2001. We are maintaining software solutions (see our full stack) with over 10 million lines of code including:

  • ERP5 - ERP/CRM/DMS/e-business
  • Slapos - Cloud Orchestration and deployment
  • Wendelin - Big Data/Machine Learning
  • Neo - Distributed Storage
  • Resist - Resilient Mesh Network
  • Renderjs - Promise based component framework
  • jIO - Virtual File System and storage connector
  • NayuOS - Private OS
  • OfficeJs - Private, offline capable productivity AppStore

Besides participating in various research initiatives, Nexedi provides customisation services for solutions with implementations being used by corporations such as Airbus, Sanef, Mitsubishi all around the world. We follow the principles of reflexive programming, enforce strict unit testing and emphasise using the latest web technologies.

Our economic model requires each developer to fulfill R&D objectives aiming at delivering short to medium sized solutions to customer requirements and progressing the evolution of our software stack. This way we try to be innovative and fund long term free software without the need for venture capital.

Interested?

We would be happy to hear from you, so drop us a line (along with your CV) at jobs(at)nexedi.com and we will get in touch with you.

Nexedi SA
147 Rue de Ballon
59110 La Madeleine
France

Phone+33 629 02 44 25
Mailinfo@nexedi.com
Webwww.nexedi.com