Sergei Volodin

First full-time engineer at the company
- Responsible for the backend development with microservices on Cloudflare Workers with TypeScript, an ArangoDB-based database, automatic data schema validation, and CI/CD with integration tests
- Conducted research into ways of obtaining data from third-party services with privacy guarantees
- Conducted analysis of the database to create better ranking results

Better recommendations on YouTube
- Responsible for the Backend (Django, PostgreSQL, OpenAPI 3.0, Spectacular)
- Machine Learning with TensorFlow 2.0, scipy, ray tune for hyperparameter search, gin-config
- system administration (Debian, Apache, Let's encrypt, Munin, firewall config, ssh tunnels between servers, e-mail, Telegram for error messages)
- parts of the front-end (React.js, Material UI)
- parts of algorithm design (online updates via Golden Ratio Search)
- parts of data analysis (Figure 8 in the whitepaper)

The code is well-tested with unit- and integration tests, and the development image uses Docker for simplicity

We collected preferences in a form of 50 000 pairwise comparisons of YouTube videos from ~2k people out of which 500 are affiliated with research institutions. The website provided search for YouTube videos with search keywords and user preferences on our criteria, such as "most clear and trustworthy video on COVID vaccines with a preference towards diversity&inclusion". The results were processed in real time (nightly) with our byzantine-proof [resilient against <51% attacks] algorithms, and are available for the general public in our browser extension. The analysis is available in our whitepaper. The project was cited by a French Military-related research institution, and is with a bigger team now

Better defenses against Adversarial Policies in Multi-Agent Reinforcement Learning
- Tensorflow 1.0 / Stable Baselines for RL algorithms (PPO), and TensorFlow 2.0 / ray tune / rllib for multi-agent support, with a TCP connection between them due to legacy support reasons
- Training the opponent and the victim in turns, with their training time (in episodes) increasing exponentially, to approach the "best response" notion from Game Theory in a computationally realistic way

We wanted to find defenses, and instead found more adversarial policies, such as agents doing "splits" and "jumps". In addition, we obtained a computational proof that the environment we considered ("YouShallNotPass") is not transitive (i.e. that there are cycles in the graph of which policy wins against another), which makes the definition of a "defense" somewhat problematic. The results, the discussion, and theoretical considerations on the topic with future directions on how to progress in this field are available in our blog post on the Effective Altruism forum

Obtaining the Causal Graph of an RL environment to increase explainability, with interventions to deal with spurious correlations

- Working on implementation and the tests (100%), theory (90%), algorithm design&brainstorming, and writing the paper
- TensorFlow 2.0 and tf-agents, with GPU training and Vizier for hyperparameter search on multiple machines in the internal cloud, and Colab for prototyping
- The results are published in a workshop paper at ICLR CLDM, and the project is continued as my Master's thesis where the causal model is non-linear, and the learning is done from raw pixels with PyTorch on GPUs

- 2017-2018: Augmented reality & robots for teaching vector addition.
- The AR library (C++, Qt/QML, shaders for speed, sensor fusion) QML-AR works on Android and uses UchiyaMarkers localize the tablet. The markers were chosen to minimize the cognitive load when looking at them
- The 3D game teaches vector addition and then tests the knowledge in a gamified format: a ship needs to reach a destination, with wind and river's current adding to the ship's velocity
- The game was tested experimentally at the International School of Geneva with promising results

- 2018-2019: Fault tolerance of neural networks
- Theoretical bound for fully-connected networks that describes the error in the output in case if the neurons randomly crash (output zero) with the necessary assumptions
- Tested on various networks with TensorFlow 1.0
The project was presented at the DS3 summer school

I extended the algorithm to search for non-convexities in images of quadratic maps to the case when the number of them is infinite. This problem has applications to balancing large power (electricity) distribution networks and is known to be NP-hard. Our algorithm extends the existing ones by allowing for higher dimension of the quadratic map. The code uses MATLAB and CVX to run a multi-step procedure involving projected gradient descent, dual optimization problems, stochastic methods and upper bounds designed by the existing team&me. The paper is on arXiv and the project was presented at a summer school

I worked in a high-frequency trading startup to reduce the latency of the existing algorithms, and to extend the operations to a financial exchange in London. To reduce the latency (which in this business determines the profit directly), I used profiling of the C++ code, tuning the networking kernel parameters in CentOS, and switching the code base to the user-space networking stack by Solarflare OpenOnload, with experiments to choose the best setup for the lowest latency, as well as tweaking the QuickFIX library to allow for faster processing. For the London exchange part, I was responsible for passing the certification and writing the FIX code for their version of the protocol, as well as implementing the trading algorithms with a console-based interface for the traders. In addition, I was responsible for performing the network dump analysis with Wireshark/tcpdump to look for inconsistencies and to analyze the performance of the algorithms.