Mike Fishy and Automation System Programming (Bots)


#61

This is exactly the problem I’m working on solving right now. Generally speaking, when the market is “alive” and tide is rising, so to speak, things don’t stop/loss out. But when market is bone dry, like they have been last several days, I find the bots are stopping out far too often and not very profitable. There’s simply not enough upward action on the prices to recoup the trading fees, so it’s best to be reasonably confident the trade, once entered has a chance to at least clear the cost of entering and exiting the trade.


#62

Love that Coin Gossip site. Exactly what i was looking for to add a bit of sentiment scoring to the mix. I’ve been toying around with http://cryptopanic.com which also has some api’s for pulling data. I’ve been working on using the voting activity as well as a basic sentiment scoring of the summary of the post to try to arrive at a basic sentiment score. I can already see a way to bring both of these sites together for one composite score.

For the CDA trending up or down, are you simply scoring off the MACD histogram or are you going by price action change?


#63

I have also been looking into incorporating the Bitcoin Futures reports.

As you may know, Futures Contracts must be reported to the CFTC.

The two reports I’m interested in are:
https://www.cftc.gov/dea/futures/deacmelf.htm
https://www.cftc.gov/dea/futures/deacboelf.htm

The idea is to take the Bitcoin Futures Contract Totals and look at the percentage going long as compared to the percentage going short. These reports are updated every 2 weeks.

I believe these Futures Contracts have a lot to do with Bear Market sentiment and even potential price manipulation as you can double your bet.

For example, if I buy $200k of Bitcoin during a dip at say $7k each.
Then wait till Bitcoin hits $8k and buy $200k of Bitcoin Futures and go short.
Then sell that $200k of Bitcoin on Exchanges driving price down for the 2 weeks…
Not only have I profited on selling my Bitcoin, I have also profited on going short on the futures contracts.

Double win. Now you know how the real mafia play the game :smiley:

Anyways, looking into adding those two futures reports to the bot to help gauge sentiment. For any manual day traders out there, these two reports would also be useful for you.

Stay Fishy


#64

It’s a custom EMA calculated within the bot itself, coupled with a custom RSI.

Stay Fishy


#65

COT (Commitment of traders) reports are super valuable and i’ve seen risk premia strategies that incorporate this as variable in their position sizing and correlation exposures on a diversified futures strategy, so for sure there is edge in it when commercial traders vs retail are positioning in ways that may signal sentiment. Also when COT reports are heavily lopsided you can hypothesize mean reverting exposures too.


#66

More Shenanigans on the Exchanges today.

I checked several Exchanges and just about all of them are seeing very small purchases of 0.000014 BTC at the market ask price almost constantly.

I noticed my bot API timeout three times already just on basic limit orders.

I’m not sure what (whoever is doing it) is trying to achieve, as I have not seen this type of activity before.

Suffice to say, I suspect this weekend may be more interesting than my analytics have suggested.

Stay Fishy


#67

Hi Team,

On the list of bots I talked about in “Are bots good or bad”, number 3 on the list is Zenbot.

The Github for the Zenbot source code is here:

It does contain some interesting strategies around bollinger bands and can be run on Windows.

While I am not a fan of Javascript, the code itself is clean and has a “models” folder where you could add your own strategies.

The main engine.js has reasonable comments that explains the thinking behind the code, including taking into account API Call lag, which is best explained in an example:

If the bot places a limit order for a CDA, it watches the order to make sure it completes within a certain amount of time, if it does not complete within that time it cancels it. However, in rare circumstances the 300 milliseconds it takes to cancel the order, the order could actually be completed and the code now needs to take care of that.

I have seen a lot of bots and code in my time and it’s when you see the author of the code putting in self recovery routines for unexpected outcomes, you know it is something that will behave in a more reliable and solid manner.

There is a lot of really good code and inspiration that can be gained from this source code on Github and it is updated fairly often (compared to other projects which rarely see updates).

Of all the other bots I listed, Gekko and Zenbot are possibly the best next to commercial options, if you are looking to get into the field of doing this yourself and need somewhere to start.

Stay Fishy


#68

Oh. Oops!

That’s my bots. I’m waging bot wars with the other bots to basically make a honeypot for them to be tied up in trying to outbid me and spend too much time processing in a loop that they don’t go after the sweet deals elsewhere.

Oh, wait, I’m not that clever, but have noticed some weirdness as well. All I can say is the period and behavior of the market we’re in these past several week just serves to shore up weaknesses in current bot implementation and that’s all I’m doing these days – one improvement after another trying to make the bots smarter, faster, and able to handle many scenarios and find more opportunities to trade.


#69

The Zenbot author is abandoning this project and starting a new bot18 project due to “design limitations” of the zenbot. One of the points he makes is that it’s no longer candles based trading:

No built-in candles or periods concept. Core does not revolve around TA and doesn’t need to build candles from raw trades (but strategies can be built that build candles and use TA).

It’s an interesting point because I definitely feel myself moving further and further away from candles and predictions based on traditional TA’s as I learn what works and what doesn’t. Nothing I’ve built is a “smashing success” as in doubling my coins every X days, but everything I’ve built works better than the things I do manually and the bots don’t sleep. That’s the main benefit I get right now with building bots.

But I am also very, very new to this market (only started learning the art of TA in October and started trading on those studies in November. But I can readily tell as I automate stuff, that bots can number crunch in a way humans simply cannot and that leads to exactly what this bot author talks about with bot18’s inspiration…ditching the candle system and focusing on order book dynamics.

This past weekend, I’ve been deep into studying order book dynamics and trade completions and learning how bots respond to my own bots. Intentionally playing cat and mouse with the bots on some of the coins that are literally doing nothing other than just sitting there. GTO/BTC and GTO/ETH pairs for example. It’s been kind of fun remote controlling other people’s bots with my own. :smiley:

And it’s also been quite revealing what strategies these bots have executed. Frankly, I’m amazed at the sophistication behind these bots and find myself learning more to the trading game than I ever knew existed!

Is there literally nothing that hasn’t already been thought up before?

Anyway, back to bot18. I want to dig more into “algorithmic trading” as I know nothing on this topic other than what I’ve learned through trial and error. Any good literature out there that is well worth the read and can translate to automation?


#70

I’ve been running Gunbot now with some level of success. The strat I’ve settled on is BBTSSL with Stochastic and Reverse Trading enabled. liking it so far.


#71

Check this out - https://bookmap.com/ and https://bookmap.com/apis/

Provides a more in depth look at order flow and you can see with a time series where people have intent of trading vs fake orders. Ie when price tests a big level where there is existing stale orders sitting, you will see how eager they are to fill at that price by watching them either pull orders or keep them with intent to fill. There is other nuances that can be discovered by watching flow as well, such as bid propping which traditional order books dont display so well. You could pull some of this data to help provide additional context or even signal that discretionary traders will never see.

You may also find this interesting too @Mike_Fishy


#72

Thanks for that Entropy, was quite interesting.

I do track order book data in my bots, specifically around volume traded as it is an important indicator.

Recently I have been adding a new strategy, which people may know as ping-pong. It is coming along nicely, but only works at very specific time periods which is quite interesting. However, this makes the 4th strategy to the bot and it is making more and more trades per day and is starting to average 800+ trades per day, which is getting impossible for my brain to track anymore.

I’m now feeding all the trade data into an Azure SQL Database and using a Machine Learning Data Science VM and getting my head around the Azure Machine Learning Studio. So far it has been a bit of a battle to get anything useful out of it, but I have a ton of Azure credits, so it’s not really costing me much.
https://docs.microsoft.com/en-us/azure/machine-learning/service/

One of the really interesting aspects is the FPGA’s:

I really like the SDK provided.

After working on a Windows App for a while, I have realised it would be better if the App was in the Cloud and usable by any device. So I stopped working on a Windows App and started deploying services in Azure to build an automated trading platform that can use AI, ML and Big Data functions in the cloud. So I havn’t been typing much on the forum lately as I am coding like a space monkey!

Stay Fishy


#73

So, after much trial and error, some success today, the bot is running in Azure in a reasonable manner.

Some things I learned in the process.

  1. Singapore is not only the fastest VM’s from my benchmarks, but also one of the cheaper regions to run systems in. It is also closer to the Exchange so latency is less.
  2. The time skew is real in the Cloud, systems time skew by up to 2 seconds, which needed me to change some of my code around the high precision timing.
  3. MongoDB is still a pain to deploy right and in a secure manner. Have to make sure you choose SSD disk or you don’t get enough I/O to do all the transactions. At least I got to upgrade to the latest version.
  4. Application Gateways are your friend, the built in layer 7 firewall is awesome and allows for custom rules, includes Dos protection as well as DDoS protection.
  5. Backups should be mandatory and encrypted, I’m surprised Azure makes it optional.

The data analytics is still going to take a while, but I got my first engine deployed on a FPGA after many unsuccessful tries. I have learned ALOT about FPGA’s LOL

Azure SQL is pretty awesome, the built in database tuning adviser and recommendations are a life saver.

Now to finish off all the book data collection and analytics, but that can be next week, going for Japanese for lunch now and will probably consume much Sake, so staying away from expensive computer stuff is recommended, but singing at Karoke is mandatory.

Stay Fishy


#74

Why do you think Singapore is faster? Just newer hardware? What is the latency btw… <10ms?

Are you running a Palo Alto L7 firewall? Im running a virtual Palo Alto VM-100 for my dedicated server in New Jersey and its been great!


#75

What is this money making bot I’ve been reading about?

I feel like I need one lol

gratz on your work!


#76

Singapore was faster based on a HammerDB benchmark. Basically I fired up the same VM size and class in several regions, ran the benchmark and looked at the numbers. I suspect Singapore was fastest as it is probably not the most sought after location for servers and thus the datacenter is probably a bit under-utilised.

For the Layer 7 Firewall, I used the Azure Application Gateway, I only need the WAF functions.

Latency was around 7ms to the API endpoint of the Exchange.

Anyways, lunch time \o/

Stay Fishy


#77

Hi all,

According to my analytics and previous market history, this week is going to be an interesting week.

Bots are all running fine in Azure, it’s nice to have it running in a proper datacenter, over running it off an old laptop at home.

My pick for this week is Ontology, it has shown previous performance at least until Wednesday. Of course, this is just my pick of the week, not any advice and be fishy on your own :slight_smile:

I’m also quite pleased with XMR today, but it’s already sold a couple of hours ago when it was almost 10% up on BTC in just 6 hours. Bots have been making better choices lately, so quite pleased with the outcome so far, but so much more to do yet…

Stay Fishy


#78

Do you have some stats with your automations

Ie W/L Ratio, Max Drawdown (for the entire system), Profit Factor, Standard deviation of returns etc - Would love to see what type of performance its doing!


#79

I have some of that on hand.

The average successful trade to stop loss trade is about 16 to 1 now after some good code changes.
The profit factor varies a lot, the running percentage is the one I use the most and averages around 600% lately. This is not a total portfolio average, it is a running tally on the trades.
An average trade is generally around $1000 but varies based on the Weibull algorithm for volume.
Returns are generally around the 2-3% mark per trade, but the system does hundreds of trades a day.

On a good day, I can get between 2 and 5 BTC, on a bad day it hovers around 1 BTC.

The system is an accumulation of BTC and BNB, I kind of gave up on ETH right now as it was being difficult. I don’t generally look at the fiat value and to be honest, the only time people should ever look at the fiat value is when they are ready to cash out, but who’d want to do that? :stuck_out_tongue:

To me, it’s all about the number of CDA’s, not their fiat value. When I get a good number of BNB, I convert it to BTC manually. When I get a reasonable amount of BTC (more than 60), I move 45 to a new Trezor, which reminds me, I need to buy another one (which will be number 3).

Some people ask how you make so many CDA’s when the market is trending down. The answer is that you can make CDA’s as long as the market is moving up and down, it is when it goes flat that it’s a problem. So why people fret over downward trends is silly, you can still make more CDA’s on downward trends by selling it when it is near the high and buying more when they are near their low points. It’s picking the high/lows which is the art of trading.

Stay Fishy


#80

Those numbers are incredible @Mike_Fishy - Seriously well done!