fairfolio
Saved Projects

Observing Behavior of Actor-Critic Driven Disturbance Agents on Cryptocurrency Exchanges

CSEF · 2019 Mathematical Sciences Third Award

Houjun Liu · Hillview Middle School, Menlo Park, San Mateo County

Overview

Objectives In this study, an example of how automated agents can potentially swing a cryptocurrency trading environment is presented. Using a group of simulated markets, reinforcement learning agents will be trained to negatively a?ect the asset s price. Simulated markets are created and populated by two class-es of agents -- the trader and the disturbance -- one trained to mimic the behavior of regular users from a target trading environment (in this case, the Poloniex Exchange), the other trained to potentially disturb the former class of agents decisions and drive the market price down. Methods First, a group of agents (the trader ) is trained simply to emulate the patrons of the real market -- a core component in building the training environment. These simple trading agents are an 8-layer-long Deep Neural Network that takes market data as input and generates a scalar that represents an order placed. During ?tting, downloaded market data and order history are split 90% to 10% for training and validation respectively. With a trained trader network, multiple instances of the graph are launched -- each acting as a separate patron in a collective network. Separate in-stances of such trained graphs will interact with each other within an emulated exchange, which serves as the environment for the disturbance agent. Next, the disturbance agent trains with the environment as highlighted above. The policy network ( actor ) takes input from the market data and creates an output scalar representing an order -- similar to the trader agent; the advantage value network ( critic ) is trained based on rewards assigned by the amount of reduction in the environment market s price. As the agent reaches a target low price, it is considered a terminal state with a reward value of 1. In a similar fashion, should the agent lose all funding provided, it is also considered a terminal state; however, a reward of -1 is assigned in this case. All other rewards are assigned based on the negative change in market price of the environment normalized between 0 and 1. Results During 10 trials, each one with 15 minutes of data collection (resulting in about 45,000-50,000 data items per trial), 9 separate distributions of trader and disturbance is tested -- increasing the amount of disturbance by 5 agents at one time. The price change (Price Delta) is calculated by dividing the change in price before and after the attack by the initial market price. The average price delta showed a decrease in market price of market price starting at a 15:35 disturbance to agent ratio; ?nally reaching the hypothesized 10% decrease in market price at a 45:5 disturbance to agent ratio.

Summary statement

Algorithmic agents can easily influence the trend of a cryptocurrency exchange if a market's volume is relatively low.

Help received

Mr. Heiko Ritter reviewed the scientific process, literature, and my work to guarantee this project's integrity. Mr. David Babington and Ms. Susan Cole helped verify that my usage of English syntax was accurate.

Awards (1)

Competition history

  • CSEF 2019 Mathematical Sciences · Entry J1410

Resources

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

Source: California Science & Engineering Fair public projects

Save projects to your library

Sign in with Google to keep track of projects you find interesting, organized into folders. An account also raises your daily allowance for “Has this been done?”, and lets you create a key for the MCP server with a much higher limit than anonymous use. Browsing stays public.

Continue with Google