Mike Fishy and Automation System Programming (Bots)


#424

I completely agree with this. Our current market shows why agree.


#425

Well guys, after all this reading I am now wiping my drives as I speak. Hopefully all goes well with reformatting after. I am taking the dive and I’m gonna start from the top of this thread and hopefully learn to code along the way…cause you all lost me about 2 thirds the way down. I figure if i start doing i will start learning.
It will be a fun winter for sure. Figure i may as well start with something that i am very interested in (CDA).
I will be trying some premade bots when they are ready, for fun, but true knowledge comes from understanding the fundamentals first.
So i hope we all keep this going as I’m looking forward to adding value to this thread (months down the raod most likely)and also learning more from it as it progresses along.

Best wishes to all for the Holidays.


#426

Holy cow, I have Ubuntu 18.04.01 installed on a VirtualBox on my MacBook Air that I’ve dedicated to this project. You were right @Mike_Fishy, I just found the Terminal App and ran the MongoDB command and boom, it installed. Thank you all, including Youtube :sweat_smile: May the learning continue!


#427

I have been messing around with Fishy’s bot here and there and this weekend, rewrote the PHP code in Ruby so I could visualize what’s going on in the bot’s logic a bit better. Although the code’s Ruby, it yields same trading results as the most recent PHP code since the logic is the same.

I’ll start with the big picture graph. I decided to focus only on BNB/BTC and BNB/USDT pairs.

The following graph is a couple of days worth of prices captured for BNB/BTC and the huge straight down drop is a point where bot was stopped and then restarted a little while later, so just pretend that a time warp happened there (no trades occurred there anyway…). Also marked on the graph is where the buy/sells executed.

The price-line is in blue, the moving averages are green for short term, yellow for medium, and red for long term. Below all are the RSI’s, again green for short, yellow for medium, and red for long.

Now what follows is the stats from each of those trades in chronological order, starting with zero:


















Sorry for screenshots instead of text, but it was fastest way for me to share this info!

There are some things I’ve noticed.

  • The bot doesn’t always buy at the lows (trades 5 and 7 and 10)
  • Sometimes entering trades where price action is decidedly sideways (trades 8 and 15)
  • Sometimes buys midway down a fairly major correction (trades, 5, 6 and 7)

Since Fishy didn’t really answer on the cycle durations, I decided to tackle this with brute force method of looping through progressively longer cycles and cycle spreads. That data follows (and it’s a lot!). Again, this is just BNB/BTC and BNB/USDT. The end result is I chose optimal values for the two pairs and above, you see BNB/BTC traded at 60/108/156.

Oh, last thing. You’ll see different sell reasons in the above individual trades:

  • loss occurs when loss threshold of -1% exceeded.
  • trailing S/L is a trailing stop loss trigger. which is set to the long trend average once sell buffer is cleared.
  • RSI is new and not in PHP script, but basically exits trade after it’s peaked and starts declining (more than 5 points).

Hopefully, this helps y’all out. For me, the bot is starting to show promise as-is, but still needs a lot of work. What worked for BNB pairs doesn’t work at all for many other pairs. I also toyed with RSI buy entry thresholds from 30 to 50 and 46 was the sweet spot (same as Fishy assigned). So now, it’s more about better detecting the bottoms or the fact that we’re in a falling trend overall (probably means factoring in BTC/USDT’s price action at the moment in time). Also, I feel like sometimes winning trades are still exiting too soon, but at the same time, the exit rules have saved many others that would’ve turned losses.

At the moment, I feel like the best place to optimize is the buy entry points as these still feel the most inconsistent of the lot.


#428

Hi all,

Looks like you guys are having fun so far and getting in practice.

I had to take a trip to Taipei for some business (buying some property), will be back after Xmas.

In which case, have a great Xmas all, see you soon!

Mike Fishy


#429

This looks already very impressive man! How about adding volume? Also price as % of 24hr high / low? This would help to determine good entry positions and exit positions i think based on mikes tips here man! :slight_smile: :slight_smile:


#430

Still at it guys. Been playing with MongoDB and Linux. Completely new to both the platform and to Mongo so lots to learn. Steadily getting to drips with MongoDB.

I am to a point where I can do simple database actions, the next task will be to read into MongoDB a currency pair. Exactly how to do this right now I am not sure but I will figure it out.

One thing that confuses me is I will need something to initialise the go fetch and then dump into Mongo. I am presuming I use Javascript for this?

What this is all about is to generate a database of pairs to start to do some stats on and to do dummy runs on.

In regards to where we go … we are going to need some sort of maths that can be slightly predictive. Use the historical indicators however not to loose sight of where it is going and not so much of where it has been.

Good effort @mwlang. I like what you are doing. I am in the realms of developing a testing base. I can see the need for several bots depending on the market conditions and some selection criteria to pick the correct one.

Crash


#431

Those are still on the TODO list. I just hadn’t gotten to it because I’ve been focused on the cycle component (duration of ST, MT, and LT). Also, not sure how order book depth gets us a reliable volume metric, so maybe someone can elaborate on that one. If it were me, I’d be reading the 24 hour ticker and using difference in volume since last tick, accounting for the 24 hour reset to zero.


#432

Mike,

I found an article that went into how the spread of orders and spread can indicate how in the very short term indicate which way things are going.

Although I am not as far along as you are, the task that I see is to get some predictive aspect into our bot. Instead of relying on historical indicators we would benefit from one that is attempting to look into the future.

This is one fantastic interesting problem I think perhaps by the end of the weekend I might have one coin pair into Mongo .

Mike , what would you use if you are coming up with a script to operate different scripts? Is this easy? The actual way to set this idea up confuses me.


#433

Share!

Mike , what would you use…

Which “Mike” are you addressing?


#434

Will do when I am home.

Mike Mwlang…

:wink:


#435

Better to call me “Michael” or @mwlang, then. :slight_smile: Otherwise, I tend to think you’re addressing our world-class famous Down Under Fishy character over there buying property in Taipei like a boss.

As a Ruby coder, I just break out logic to different object classes to handle various scenarios and switch those scenarios given current market conditions. In other words, instead of a bunch of if/elsif/elsif/else conditional branching all over the place. In other words, Gang of Four’s State Pattern: https://en.wikipedia.org/wiki/State_pattern which is effectively the “context” for their other pattern, the Strategy Pattern: https://en.wikipedia.org/wiki/Strategy_pattern

So the State pattern is a collection of classes, BuyReadyState, BuyPrepState, BuyOrderState, etc., as well as perhaps other market state classes, like say BtcTrend, RsiTrend, VolumteTrend, alongside the CDA instances with accessors to current prices, ST, MT, LT trends, and so on and that is the “context” to the strategy classes that can decide to advance the states and/or execute further actions (i.e. buy or sell) and you can create different strategies for different market conditions.

The same thing can be done with PHP classes and what you do is treat the main loop that fetches prices chooses the trading strategy and then goes through the trading strategy logic as the “engine”. It’s logic doesn’t change, but the code that makes the decision to buy/sell can be swapped out at any time, given different market conditions.

I dunno if that helps or if it’s a slight breeze over the head –

I do know it’s a densely worded couple paragraphs, so just ask for more details if needed.


#436

@mwlang

A little bit confusing tbh. What I am wondering is can we have something like VBA where we can make individual functions called by other functions or macros? For example we can write a function called RSI and then call that function into a calculation which spits it out accordingly.

The order book reference I found is here:

Shows promise at predictive output. However, I have been looking at various predictive references, they are all a bit hit and miss tbh.

I think from what I can visualise it will be best to use Python with MongoDB. There is lots of code out there with Python already that I can adapt.

Does this State “stuff” all run from one script? This is a bit confusing.

Cheers @mwlang

Crash


#437

@Crash101, maybe this video helps. It’s a philosophical approach to coding using object oriented approach:

Although, not mentioned specifically what I was saying above, it’s all very much in the approach to writing the code. And since PHP has classes and object oriented programming constructs, everything discussed in the video can apply equally to PHP and to Python for that matter (which is also object-oriented).

As for “all in one script” – I rarely ever write just one big script in a top-down fashion. I usually break things out logically into many files and include them as needed in the main script (or other files as appropriate). For now, since what we have is such a simple script, it hasn’t quite reached the point of breaking things out. However, the last version I submitted, if this were a bigger project, the CDA class and the Trend class would reside in their own files, “cda.php” and “trend.php” and thus would begin the break away from one big script.


#438

Hi all,

Merry Xmas to you all and hope you have a great new years eve smash :smiley:

To take full advantage of your computer, think a little differently to just one program.
Have many programs that work together using a single data source (such as Mongo).

For example.

One program just fetches data, checks the sanity and writes it into Mongo with a Mongo “Collection” per pair.
In MongoDB, a “Collection” is like an individual database.

Have a second program that fetches the 24hr stats and writes the stats into Mongo

Have a third program that collects the order book data, sanitise and process it and write that to Mongo.

Fourth program reads all that data from Monog and calculates ST, Mt, LT and RSI and writes those stats back to Mongo.

Fifth program reads those stats and uses K-nn algorithm to predict potential future trends, writes its predictions to Mongo, then gets what the cycle actually was, feeds that result into the learning and repeat, each time looking for the least drift between prediction and result.

Sixth program uses all the calculated data and K-nn to find entry points and to spawn N+1 trade processes which handle the orders.

Basically, all completely separate programs that run individually on a multi-core processor, doing each of their jobs like a production line.

After it gets bigger and more complex, you will find this method easier to maintain later, than one big program trying to do it all, plus it is more efficient and makes better use of the CPU.

The down side, if one program fails, all the others need to be “informed”, which I usually use an Ambulance process and flag files (a file that exists only when a problem has been encountered, plus lots of logging to the problem file until it can either be rescued automatically, or give you lots of useful info to know what went wrong).

One program is like just one Fishy.
Many programs are like a school of Fish each helping each other to survive the swarming sharks :stuck_out_tongue:

Stay Fishy


#439

@ Mike_Fishy and @mwlang

thanks for these two excellent posts. The idea of trying to do it all in one was doing my nut in. I like the idea of breaking it up into steps. This makes sense to me.

I have a beast of a machine with Linux installed. It should be able to handle the multiple program approach no problem.

I wish you all a very good remaining festive holiday period!

Crash.


#440

Microservices is what Fishy describes and it’s powerful stuff:


#441

Hey everyone,

I have been thinking about how to do this quite a bit and been doing quite a bit of research. For me, it will be easier to programme this stuff with python as I can find bits of code already completed for various parts.

In addition to using python, I have come across something called Anaconda. It downloads a whole bunch of stuff on your computer including a Datatable system called Pandas. It seems to be easier for me anyways to use as it is designed for data manipulation and calculations.

With Anaconda there is a shed tonne of references to do with ML and other really cool stuff. i am guessing once we have the basics working and happy with it this is probably the way forwards.

I have found lots of free resources with lots of code already written in python. It great to have free code however, the trick will be to only use the bits I am interested in.

Hope this strategy makes sense. its a way to handle the data. May not be as quick as Mongodb but i think it gives me a higher chance of success initially.

Crash


#442

Just a quick post to all you potential bot testers and programmers… I have a couple of testing scripts running on my PC at the moment… keep them running and see the percentage keep increasing…
image

Keep the focus… this is money making… keep going!! motivation…

Crash.


#443

What are your stats looking like so far?