Deep Learning with Forta

Forta is a real-time detection network for security monitoring of blockchain activity. The decentralized Forta Network scans all transactions and block-by-block state changes, leveraging machine learning to detect threats and anomalies on DeFi, NFTs, bridges, governance and other Web3 systems. When an issue is detected, alerts are sent to subscribers of potential risks, which enables them to take action.

Machine learning is becoming a crucial tool in any defender’s tool box. Over the last year, Forta has rolled out support for a broad range of machine learning techniques. Numerous bots deployed on the Network today leverage machine learning to assess blocks and transactions for malicious activity in real-time: anomaly detection to identify novel attacks, time series analysis to sight price manipulation attacks, clustering to group related attacker addresses all the way to supervised models to alert on malicious smart contracts. The success seen through machine learning is resulting in the community continuing to innovate in this space. 

This blog showcases the latest innovation of applying deep learning to threat detection, diving into a recently deployed bot: the sentiment analysis bot. Before going into the details of this bot, let's spend a bit of time on the background of why deep learning is such an exciting technology.

Deep learning had its breakthrough moment in recent years. We all now use some type of deep learning application on a daily basis. Be it in voice activated agents or translation services which all have been made possible through deep learning. This last year, deep learning has reached the masses with chat bots, like ChatGPT or image generation models like Stable Diffusion (the image on the below was generated by Stable Diffusion and the hero image was generated via DALLE 2). Content generated by AI has become so realistic that humans can no longer differentiate AI from human generated content. What is possible today sounded like science fiction not too long ago and is now available to all in a matter of a few mouse clicks.

Prior to deep learning, machine learning utilized carefully curated labeled datasets. Data scientists, through analysis and intuition, engineered features that they believe would be helpful for the algorithm in the task it was assigned. Deep learning operates differently. It can train on vast amounts of unlabeled data using self-supervised techniques. 

Imagine a model to predict the color of animals. One could create a dataset of animals’ features (size, has fur, lives in africa, etc.) and colors. In a self supervised approach, however, one can feed in vast amounts of text (unlabeled data) and omit parts of the text the model needs to predict.

This technique allows the model to learn concepts like the elephant is grey, lives in the African savanna, etc. It can learn the color of animals without being explicitly taught. As you can imagine, this is very powerful as vast amounts of data can be utilized by the model in a manner that is less biased than the original approach of curating a dataset, feature engineering, etc.

The advantages of deep learning span to security as well. For instance, in the context of a malicious smart contract model, a data scientist may decide that the number of addresses hard coded in a contract could be a feature to differentiate malicious from benign smart contracts. However, the data scientist introduces bias to the model and throws away vast amounts of unlabeled data with this approach. Deep learning can do much better as shown by the following bot:

Sentiment Analysis Bot

The sentiment analysis bot is used to decipher on-chain text messages. On-chain text messages are special transactions that one can issue on-chain that contain text in the input data field of the transaction that can be used to communicate with wallet owners. In the context of security, they can be used to warn users of scams or convey discontent with an attacker, like the example shown below.

While one can use specific keywords to identify these messages, the ability to do so with keywords is unlikely to be robust. Different spellings/language will cause messages to be missed. Also, a keyword approach doesn’t really tell you the difference between “This is a scam” vs “thanks for alerting me to this scam”.

Deep learning works very well on natural language text. The advantage of deep learning is that models that may have been trained on large corpus of publicly available data (e.g. web data) can be transferred into a different context, like on-chain text message classification. This allows one to utilize an existing trained model as is being done by the sentiment analysis bot, which uses two pre-trained models:

- Twitter-roBERTa-base for Sentiment Analysis - UPDATED (2021)
- Emotion English DistilRoBERTa-base

The sentiment analysis bot feeds each text message into the above two models to classify positive/negative sentiment and an emotion. This bot can detect emotional sentiment of anger, disgust, fear, joy, neutral, sadness, and surprise. The classified text messages can then be used to identify scammers and scam projects.

An example alert output of the bot is shown below:

In order to utilize deep learning in one's code is trivial, thousands of open source libraries are available to use across varying applications via heavily documented code. The main code of the sentiment analysis bot consists of 144 lines of code as shown below:

The first few lines import the transformer library (line 7) and define the respective models (line 20-23), which have been packaged with the bot. The initialize function loads the models with a specific configuration for the task at hand (line 29-47).

Sentiment analysis is then performed in this code. In line 100, the text is extracted from the transaction (if there is any) and passes it into the sentiment and emotion model on line 106 and 108 respectively. The remainder of the code then wraps the sentiment and emotion into a finding object that is emitted by the bot. 

As seen, utilizing deep learning can be straightforward and powerful. This enables the defender to move away from brittle heuristics and use off the shelf pretrained models. You can now easily package the model and inference code as part of your bot, deploy it, stake it, and your model will start monitoring/classifying transactions and blocks in real-time. The Forta Network is structured in a way that inference can be distributed across several nodes utilizing sharding and ensuring each transaction/block is processed reliably. 

This blog showcased how to utilize deep learning in the context of security and hopefully inspired you to add deep learning to your toolbox. Be sure to share what you are working on the Machine Learning channel on Forta’s Discord and head over to Forta’s ML docs page when you are ready to get started building!