Mike Fishy and Automation System Programming (Bots)


#129

Oh, I see! I was totally going down a different road thought-wise when you mentioned “milestones” It’s more like a mutex/semaphore on the CDA itself to ensure both correct bot is working and no two bots crossing paths on any one CDA at any given moment.

Thanks for clearing that up!


#130

Seems to be coming along nicely now, need to fix a couple of little bugs and some rounding errors :stuck_out_tongue:

Iteration = 4260
Current BNB Price is: 12.90000000 T:DOWN
Running Time: 375.2 mins
Running Strategy: Traffic Lights
AutoBot BNB Made: 65.204330400001
Binance BNB Fees: 15.4548848
AutoBot running Percentage: 59.38

At least it is making me more than it is making Binance now \o/

Stay Fishy


#131

So, I left the new bot running last night (which was a risk in itself) and it created quite the mess.

grep Buy EMAT.* |wc -l
1966

Did a total of 1,932 trades which completed successfully and 34 which it totally stuffed up on.
Not bad odds considering the code is only a week old and up to version 88.

So had to do a cleanup this morning and stopped all the bots early to schedule the analysis.
I now have my weekend work cut out to try and figure out where it went wrong on those 34 failures.

However, even with the ones it stuffed up on, the bot made 2,448.77 BNB after fees (about 5 BTC)
Binance made 705.43 BNB in fees, they must love my new bot LOL

This bot only works well when the market is sideways, so going to update the Monarch process to track things better as the market started dumping again (must be Friday \o/)

Stay Fishy


#132

Nothing like testing with Live Data and Real Trades to find and iron out those bugs!

Due to a logic error in the bots, they lost about 7 BTC.

Oh well, easy come, easy go :slight_smile:

On a more positive note, the bug is fixed and it is making that loss back pretty quickly :hugs:

However, I also updated the Monarch process to be clustered across three machines in three different datacenters now to give the service better redundancy using ZooKeeper. Working on the other processes as well now that I have a good clustering solution.

Anyways, still so much to do to bring this service to life and I am working on it and setting my stretch goals, the list which keeps on growing.

Stay Fishy


#133

Updated the new bot with LeastSquares Linear Regression.

Code was surprisingly simple.

Example:

$samples =  [1532035319131, 1532035319231, 1532035319331, 1532035319431]; // Unix Time
$targets = [[60], [61], [62], [63]]; // Prices
$regression = new LeastSquares();
$regression->train($samples, $targets); // Train Algorithm with the data

Once the algorithm is “trained” with data, you can use:

$regression->predict([1532035319931]);

Which will return the estimated price based on LeastSquares Linear Regression at the Unix time specified.

So far it has proven to be quite accurate and useful.

Also looking into the NaiveBayes Classifier, but looks only to be useful on cross-trades.
ie: the likelihood of an event sequence.

On a more positive note, the bugs and math/logic fails in my code have been mostly eliminated now :slight_smile:

Still so much to do, think I underestimated the effort needed, should have used a multilayer perceptron classifier to predict the work needed :stuck_out_tongue:

Stay Fishy


#134

Are you human or are we all talking to AI ? :wink:

Do you find regression works better during high momentum periods or slower periods?


#135

Hmmm… I can see finding a slope in noise using this to get slope indicator on price change using this…but surely it doesn’t trump K-nn – or are you applying least squares linear regression in a different way than predicting future prices?


#136

Using K-nn on the short EMA, the linear regression on the long EMA, while the middle EMA is the indicator.
It’s a part of the Traffic Lights strategy.

Edit: Middle EMA is now being used for the RSI calculations

Stay Fishy


#137

Created a 4th EMA, moved RSI to the 4th EMA running at a longer term, provided for better input on switching.

Volume is almost non-existent lately, wonder where everyone has gone?

Profit margins have really dried up as well, lucky to pull 0.6% when being aggressive and given the really low volumes, it is not amounting to much at all.

One thing I will say, anyone that thinks you can buy a bot off the market and just run it, is deluded.
It needs constant attention (almost daily), things appear, disappear and change constantly on the exchanges, it is often difficult to keep up. Even the API call that says what decimal places to use, is not being updated to reflect reality of what is actually being used. I switched to querying the book orders to figure out how many decimal places to use now as it changes constantly…

Anyone that has seen or used a Forex bot will know they are text book and run for years without needing changes. Not like new countries spring up out of thin air and create a new Forex currency every day. The CDA space is completely different and sometimes quite frustrating and doesn’t look like that is going to change anytime soon.

Anyways, as always - Stay Fishy


#138

Damn when I read this thread …and its probably my favourite…
I feel im reading Albert Wesker’s Diary!! :joy:

Glad your on top of things Mr Fishy. I dont think I make 1900 trades in 8 years!
I shall stay tuned for the next episode. ATB.
:beers:


#139

It sounds like managing this is a full time job. Realistically, are you ever afraid the bot will make a series of trades when you are asleep that will ruin your profits or portfolio?


#140

As Mike stated, when he has 45 BTC or so in profit, he move them to a cold storage. So I guess he’s pretty safe :stuck_out_tongue:


#141

No, the bots can’t ruin me, they have a set maximum limit of 15 BTC.

Also, been there, a logic error lead to some losses not long ago, also when starting out, I was not ready for the pump and dumps and other shenanigans, which also lead to some losses. I thought it would be just like Forex trading right? Wow, was I wrong.

Yup, as soon as it hits 60+ BTC, I move 45 BTC to a Trezor, working on Trezor number 3 now.

I didn’t read all that B90x stuff right, I thought Peter said you needed 90 of em! :joy:

Stay Fishy


#142

I’ve noticed this as well and been working hard at building different strategies to try to turn profits even in this stagnant market. Make no bones about it…it ain’t easy by any stretch of the imagination. This may sound weird, but I am kind of getting further and further away from going on traditional TA style probability and predictive analysis with candlesticks and focusing strictly on the order books and the rolling trades log and coupling that with signals coming from the surrounding markets and even other exchanges. Mostly I’m looking for patterns in either the size of the orders coming through and how they change the direction of the price action and clustering of orders around certain points as a way to form some sort of predictive indicator of what’s to come next…if nothing else, during this dry spell, I’m taking time out to build visual heads up displays on all that data that’s flowing through the pipelines. One I’m lately truly fascinated by is the trades log activity as seen below. The bigger the trade size, the bigger the bubble. The higher the price, the higher up on the Y axis.

oh, and while I’m typing this, I see Fishy’s 37 bitcoin order (along the way to reaching 60 flow though…)

I have to say, @Mike_Fishy is absolutely spot on that just building a bot or even a few bots and just letting them run maintenance free is NOT the reality. I am constantly tending to and improving mine as well. If nothing else, I’m learning a ton about what’s really happening behind the scenes that simply isn’t exposed to most people trading these markets. Capturing data, analyzing and trying to build bots to play into those trends seems to be a never-ending one-up-the-other-bots game.

I actually started wondering if I was doing things the hard way and went and grabbed a couple of the open source bots off the shelf and fired them up…the one I tested most was Zenbot and boy is that thing full of bugs. It seems to miraculously make profit when in simulation mode, but the moment you go live with it, it’s all losses. Maybe I just had the knack for turning it on at just the right time to realize those losses but over a couple day’s experimenting, I have to say I’m simply not impressed.


#143

LOL, stop pointing out my market manipulations :stuck_out_tongue:

Always be Fishy!


#144

Have you found these algos effective on traditional markets at all or is it because Crypto is so nascent and the lack of institutional order flow means pricing inefficiencies are common?


#145

Forex bots are much easier and only need simple EMA/RSI as there is no volume issues and the market is reasonably stable and easy to deal with. The fixed rate fees also make it easier, it doesn’t change much and decimal places are known and stable, but you need to work will much larger amounts of funds, so not affordable for the average person.

Must admit, the Forex bot has made significantly more than the Crypto bot and needs far less attention.

However, I wanted to experience the “wild west” as people called it and see just how smart you can make a bot that deals with all the stuff that goes on. It is actually much harder than I thought it was going to be and the code just keeps growing and needs a more powerful computer to run it now than the one I started with. I can say, it’s not for the faint of heart to be here.

One thing I have learned in my many decades, life revolves around risk and reward. The more risk you are willing to take, the higher the potential reward, but also the greater the potential fall. Risk adverse people will probably stick to traditional markets. The real risk takers, willing to go out on a limb and make the most out of life, are the people who truly live - win or not, you can’t say you didn’t live on the edge!

Stay Fishy


#146

Who wants to live a life and one day think what if? Might as well give it a good crack, just dont pour all your eggs into that basket!


#147

Seeking a little help here on algos that can help me buy nearer the lowest lows and highest highs when price swings in reversals.

Up until recently, I mostly used candle stick data and order book data to make the calls on buying and selling, but I’m leaving close to 30% of the potential net profit on the table with each swing, so I’m trying to figure a better buying and selling algo…mostly I’m focusing on the buying as trailing stop losses that get more aggressively close to the price action works well enough for the moment.

Anyway, to cut to the chase, I feel the next step is incorporating the live trade action into the decision making and I now have live trades feeds downloading smoothly with low latency and I’m trying to come up with a way to tell just from the flow of trades when a low price is likely reached. I am getting good enough at getting within range to buy. That is to say, I’m good at figuring out when the low is imminent to within 1 to 2 minute span, but I’m not doing so well guessing the bottom number and prices tend to retrace a good bit before I make the buy decision.

Here’s a good example from BNB/BTC just in the last hour…my bot called it right and got ready to buy, but missed the lows twice in a row by whopping 0.9% (approx):

What do you guys do to get your buy orders filled as close to the local lows as possible?


#148

My bots don’t use candlestick data. It fetches the current price (last completed order) every 2 seconds and loads that data into arrays. My bot can track it to the bottom, identify the bottom and trend reversal at much finer granularity. That’s why it is so aggressive. Comes at a higher risk though.

To help mitigate some risk, I include a buy lag, which is based on tracking all CDAs that are dumping. This lags buy orders to follow other bots in the system that are trading on other CDAs. When the trend reversal is indicative across the spectrum, it is more likely to be a good time to buy in.

Candlestick data at 1 minute is not as effective as price data every 2 seconds. Bot needs a lot more memory to do it, but works quite well when done right.

However, I am going to say, it does increase your risk levels.

When thinking about it and looking at charts, put a piece of paper over the part where you can see the trend reversal, so you can’t see it. Then start asking, how do I identify when the trend reversal is going to happen (when you can’t see it)?

The EMA lagging indicators wont catch the bottom, but do give better indication of a trend reversal, but won’t make you as much. So now you are going to be “guessing” the trend reversal, which is why it carries more risk.

Switching from candlestick data to rapid price data will make a big difference though, even with your EMA calculations as they will be more granular.

Stay Fishy