Mike Fishy and Automation System Programming (Bots)


#121

Crypto is Crazy

So technically, trading two supposed “stable coins” against each other where both are supposedly backed by the USD, makes the USD a Forex to the USD and volatility occurs.

Is it just me, or is this complete craziness? :stuck_out_tongue:

Stay Fishy


#122

Added a new strategy to the bot today, so far the code seems stable.

The new strategy is called Traffic Lights and is a bit complex, but a very aggressive bot and is doing quite well so far. Its been running for about 3 hours and has done over 100 trades already, but really small percentages, so its only managed to turn the 2 BTC I gave it into 2.39482 BTC.

Still was an interesting new strategy to add to the collection and my source code now just exceeded 800Mb.

The market volatility in this “trade war news hype” period has been awesome, things are all over the place. Shall be yet another interesting weekend for the analytics.

Stay Fishy


#123

Fixed a metric ton of bugs in the new strategy code and added some new features.

It now reports how much I’m making Binance in fees since the new strategy can do just over 1000 trades a day due to it’s excessive aggressiveness. Seems Binance is starting to love me with how much the bot makes them LOL.

Anyways, back to the grind, need to complete a few more tasks before turning in.

Stay Fishy


#124

@Mike_Fishy Are you planning to create a buyable product from your bot with different setups? If it has a GUI which can be interpreted by ordinary people it would have a good market I think. At least I would probably buy a copy of it with a basic setup for training purposes. :star_struck: Maybe I would try to use it without GUI too.

Sorry if this question came up before.


#125

So, what is this new aggressive strategy? “Traffic Lights and is a bit complex” is kinda obtuse, ya know? :slight_smile:


#126

The new strategy works by tracking over 1800 trading pairs and pre-defined milestones.

As each CDA reaches certain milestones, it preps the bot to rapid trade it in succession, generally pulling 0.35% at a time, which after fees (0.1% buy and 0.1% sell), results in 0.15% trades.

Still very much in testing stage and it takes about 1.5 hours to start up before it fires off rapid trades, but basically looks like this (this is the BNB one, there is separate ones for BTC and ETH as they need to all run in parallel and talk to each other for cross-trade opportunities).

TL

So yes, a lot more complex as the 3 tasks also open socket connections with each other to communicate milestone data, while a 4th process (the monarch) controls the traffic lights (stop, start, idle).

When coding such bots, need to stop thinking like a human and start thinking like a parallel computing platform (or a team of humans that need to work together for a strategic goal).

However, in terms of the milestones, there is basic rules at the moment which I am expanding upon.

All still very much in testing stages and so far it seems to make Binance more money than me LOL.

Stay Fishy


#127

Fascinating stuff. I get the concept of multiple bots talking to each other for cross-trade opportunities. That’s smart and a good way to segment logic. What I don’t quite get: What constitutes a milestone?


#128

The milestones are broken down into functions that cause the decision making of the bots.

  • Buy Ready - there is no current orders on the DCA by any of the processes and is available in the pool of CDA’s to trade.
  • Buy Prep - the TA has decided that the CDA is heading to a buy potential
  • Buy Order - the CDA has reached the buy potential and the bot buys some and signals the other bots it is active on the CDA or if cancelled, releases the CDA back to the Buy Ready milestone state.
  • Buy Completed - the buy order has completed or adjusted to get completed - order management to get the order completed. Signals other bots it has bought the CDA with quantity and estimated value.
  • Sell Ready - any of the bots look for the TA milestones that the CDA purchased is headed for a sell opportunity.
  • Sell OK - one of the bots has identified the CDA has reached a sell potential and signals the other bots it is active on the CDA.
  • Sell Order - the sell order has been completed or been cancelled or adjusted to get completed - order management stuff is done. If cancelled, places CDA back to the Sell OK milestone state.
  • Sell Done - marks CDA to the Buy Ready milestone state and perform calculations on how much fees were paid, the volume of CDA that was processed in the trade, the amount of BNB, ETH and/or BTC made and the running percentage.

Edit: there is a few other milestones as well, but you get the jist of it.

Edit: quick example I just pulled off the bot logs:

==========================
Runtime 89.63
Buy Limit order placed on IOTABNB at 0.08370000 with quantity 1314
Array
(
    [symbol] => IOTABNB
    [orderId] => *edited out*
    [clientOrderId] => *edited out*
    [price] => 0.08370000
    [origQty] => 1314.00000000
    [executedQty] => 0.00000000
    [status] => NEW
    [timeInForce] => GTC
    [type] => LIMIT
    [side] => BUY
)
==========================
==========================
Runtime 92.12
Sell Limit order on IOTABTC placed at 0.00015523 with quantity 1313
Array
(
    [symbol] => IOTABTC
    [orderId] => *edited out*
    [clientOrderId] => *edited out*
    [price] => 0.00015523
    [origQty] => 1313.00000000
    [executedQty] => 0.00000000
    [status] => NEW
    [timeInForce] => GTC
    [type] => LIMIT
    [side] => SELL
)
==========================

Quite Fishy


#129

Oh, I see! I was totally going down a different road thought-wise when you mentioned “milestones” It’s more like a mutex/semaphore on the CDA itself to ensure both correct bot is working and no two bots crossing paths on any one CDA at any given moment.

Thanks for clearing that up!


#130

Seems to be coming along nicely now, need to fix a couple of little bugs and some rounding errors :stuck_out_tongue:

Iteration = 4260
Current BNB Price is: 12.90000000 T:DOWN
Running Time: 375.2 mins
Running Strategy: Traffic Lights
AutoBot BNB Made: 65.204330400001
Binance BNB Fees: 15.4548848
AutoBot running Percentage: 59.38

At least it is making me more than it is making Binance now \o/

Stay Fishy


#131

So, I left the new bot running last night (which was a risk in itself) and it created quite the mess.

grep Buy EMAT.* |wc -l
1966

Did a total of 1,932 trades which completed successfully and 34 which it totally stuffed up on.
Not bad odds considering the code is only a week old and up to version 88.

So had to do a cleanup this morning and stopped all the bots early to schedule the analysis.
I now have my weekend work cut out to try and figure out where it went wrong on those 34 failures.

However, even with the ones it stuffed up on, the bot made 2,448.77 BNB after fees (about 5 BTC)
Binance made 705.43 BNB in fees, they must love my new bot LOL

This bot only works well when the market is sideways, so going to update the Monarch process to track things better as the market started dumping again (must be Friday \o/)

Stay Fishy


#132

Nothing like testing with Live Data and Real Trades to find and iron out those bugs!

Due to a logic error in the bots, they lost about 7 BTC.

Oh well, easy come, easy go :slight_smile:

On a more positive note, the bug is fixed and it is making that loss back pretty quickly :hugs:

However, I also updated the Monarch process to be clustered across three machines in three different datacenters now to give the service better redundancy using ZooKeeper. Working on the other processes as well now that I have a good clustering solution.

Anyways, still so much to do to bring this service to life and I am working on it and setting my stretch goals, the list which keeps on growing.

Stay Fishy


#133

Updated the new bot with LeastSquares Linear Regression.

Code was surprisingly simple.

Example:

$samples =  [1532035319131, 1532035319231, 1532035319331, 1532035319431]; // Unix Time
$targets = [[60], [61], [62], [63]]; // Prices
$regression = new LeastSquares();
$regression->train($samples, $targets); // Train Algorithm with the data

Once the algorithm is “trained” with data, you can use:

$regression->predict([1532035319931]);

Which will return the estimated price based on LeastSquares Linear Regression at the Unix time specified.

So far it has proven to be quite accurate and useful.

Also looking into the NaiveBayes Classifier, but looks only to be useful on cross-trades.
ie: the likelihood of an event sequence.

On a more positive note, the bugs and math/logic fails in my code have been mostly eliminated now :slight_smile:

Still so much to do, think I underestimated the effort needed, should have used a multilayer perceptron classifier to predict the work needed :stuck_out_tongue:

Stay Fishy


#134

Are you human or are we all talking to AI ? :wink:

Do you find regression works better during high momentum periods or slower periods?


#135

Hmmm… I can see finding a slope in noise using this to get slope indicator on price change using this…but surely it doesn’t trump K-nn – or are you applying least squares linear regression in a different way than predicting future prices?


#136

Using K-nn on the short EMA, the linear regression on the long EMA, while the middle EMA is the indicator.
It’s a part of the Traffic Lights strategy.

Edit: Middle EMA is now being used for the RSI calculations

Stay Fishy


#137

Created a 4th EMA, moved RSI to the 4th EMA running at a longer term, provided for better input on switching.

Volume is almost non-existent lately, wonder where everyone has gone?

Profit margins have really dried up as well, lucky to pull 0.6% when being aggressive and given the really low volumes, it is not amounting to much at all.

One thing I will say, anyone that thinks you can buy a bot off the market and just run it, is deluded.
It needs constant attention (almost daily), things appear, disappear and change constantly on the exchanges, it is often difficult to keep up. Even the API call that says what decimal places to use, is not being updated to reflect reality of what is actually being used. I switched to querying the book orders to figure out how many decimal places to use now as it changes constantly…

Anyone that has seen or used a Forex bot will know they are text book and run for years without needing changes. Not like new countries spring up out of thin air and create a new Forex currency every day. The CDA space is completely different and sometimes quite frustrating and doesn’t look like that is going to change anytime soon.

Anyways, as always - Stay Fishy


#138

Damn when I read this thread …and its probably my favourite…
I feel im reading Albert Wesker’s Diary!! :joy:

Glad your on top of things Mr Fishy. I dont think I make 1900 trades in 8 years!
I shall stay tuned for the next episode. ATB.
:beers:


#139

It sounds like managing this is a full time job. Realistically, are you ever afraid the bot will make a series of trades when you are asleep that will ruin your profits or portfolio?


#140

As Mike stated, when he has 45 BTC or so in profit, he move them to a cold storage. So I guess he’s pretty safe :stuck_out_tongue: