Mike Fishy and Automation System Programming (Bots)


#182

@Mike_Fishy

I will have a go at this, this evening. My ability to code in VBA is pretty good. Very interested in making a bot. Have done some stuff with databases as well.

Awesome. Thanks for your step by step guide. Major Qudos.

Crash.


#183

@Mike_Fishy
Thank you for this. Now I will earn something new! Btw I can write (“code”) only HTML, CSS, little javascript so I hope it will be enough to follow the steps. I will soon start to learn python and later step forward maybe towards php.


#184

@Mike_Fishy Thank you for the Awesome guide and I am Mechanical Engineer and never done coding before but your guide was great and easy to follow.
I got passed step 3 and when i execute the step 4 command i am always getting the following error.

Tried to install manually by downloading conposer.exe y and still getting the same error.Checked the ini file the path is set correctly and not sure how to get past this error.

Thanks in Advance…


#185

Make sure the php.ini has the line above uncommented.
Also, have a look in the “C:\php7\ext” folder and make sure all the dll files are there.
If not or you are not sure, delete the C:\php7 directory and start again.

Looks like a few of you have been able to get the framework running, so lets start with a few programming concepts.

Firstly, there is two things we will be using in our bot to keep our data.
These will be variables and arrays.

If I explain an array first, then the explanation for the variable will be easier.

An array is like a spreadsheet, it has rows and columns where we can put data.
So if I were to define an array like this:

$myarray = array();

That creates an area in memory, just like a spreadsheet, but it is blank and contains no data.

A Variable on the other hand is the same concept, but it only has one bit of memory to store just one piece of data. So I could define a variable like this:

$myvariable = "This";

So that area of memory contains the word “This” and we can use and process this data.

So, just remember, a variable holds just one piece of data, whereas an array is more like a spreadsheet and can contain many pieces of data in rows and columns. The most common use for an array is storing data in just one column, such as I can store price data I collect with each price placed in the first column, but many rows of the price data.

So, as an example, lets say I was interested in tracking the price data of ICX traded against USDT.

In our example done yesterday (above) where I used a function to get the prices:

$p = getprices();

The $p in this example is an array of data. It is like a spreadsheet with two columns.
The first column we call the “key” and it contains the name of the trading pair.
The second column we call the “value” and it contains the price.

That gets the prices for every trading pair on Binance, it is quite a long list.

So I loop through the list looking for the one I want like so:

foreach($p as $key => $value)
{
  if($key == "ICXUSDT")
  {
    // do something here
  }
}

The “// do something here” is a comment in the code that does not get processed. It is to remind ourselves about what our thinking was at the time.

The “if($key == "ICXUSDT")” is basically filtering through the list to check if the row contains the trading pair we are interested in and if it does, then we do something with that.

So, the $key is a variable which contains the trading pair name and the $value is a variable which contains the price for that trading pair.

So if we just get the price once and then filter for the ICXUSDT pair, we can put the price into our own array and then use that array of prices to perform logical decisions and actions. However, we have only put in the price once, so not much use right now.

So the next thing is we want to get the price several times over a period of time.

Enter the next concept, it is call looping.

A simple loop looks something like this:

for($i = 0; $i < 200; $i++)
{
 // Loop 200 times doing stuff here
}

So now we can use a loop to get the prices 200 times and put each price into our array, so the array will contain a list of 200 prices we collected.

So if I create an array for our prices and call it $icxtrend like this:

$icxtrend = array();

Then I collect the price using our “foreach” and filter loop, I now have the price in the variable called $value.
So I can “push” that price into our array like this:

array_push($icxtrend, $value);

The first time I push the value into our array, it will be the first and only item in our list. If I get the prices again and then “array_push” again, the new value is the second item in our list of prices… and so on…

Now, lets say I get to 200 in our loop and now have 200 prices in my $icxtrend array.
I don’t want to stop here because I want to calculate a moving average for the price.
However, I don’t want to have any more than 200 prices in my array, so I can calculate a 200 price period moving average.

So, once we get to 200 prices, the next price we get, we want to do two things.

  1. Get rid of the oldest price and shift all the prices up one row in the spreadsheet.
  2. Add our new price to the bottom .

This is easy in PHP as there is a command we can use called “array_shift”.
This command shifts the list of prices in our array up one place, removing the top one.
So just like in a spreadsheet, where I click the top row and select delete and the spreadsheet asks me to shift the rows up, this command does exactly that.

So, after collecting 200, I want to array_shift, then array_push like this:

array_shift($icxtrend); // deletes the top row and shift all the price up one place
array_push($icxtrend, $value); // adds new price to the bottom

Now we have these basic concepts in mind, give me a moment and I will put something together where we can use this information and do some logic.

Let me know if I am explaining things ok or if there is anything you need clarification on. It can take some time to get started in programming, but once you get use to the concepts, these concepts are the same in almost every programming language, just the command syntax is different.

Next post, I will place example code of tracking a 200 period moving average on ICXUSDT using the concepts I explained above, but the loop wont stop at 200, we will let it run over 2000 periods and display some useful information we can use for the next part of the logic.

Stay Fishy


#186
<?php
print "Load the Biance API Framework\n";
require 'vendor/autoload.php';

print "Initialize the variables and arrays we use\n";
$i = 0;
$binance_prices = array();
$icxtrend = array();

print "Create a function that gets price data from Binance\n";
function getprices()
{
  $api = new Binance\API("<api key>","<secret>");
  $mp = $api->prices();
  return $mp;
}

print "Starting our Loop for 2000 iterations\n";

for($i = 0; $i <= 2000; $i++)
{
  $binance_prices = getprices();
  foreach($binance_prices as $key => $value)
  {
    if($key == "ICXUSDT")
    {
      if($i < 200)
      {
        print "Iteration number $i in our loop got the price $value\n";
        array_push($icxtrend, $value);
      }
      else
      {
        print "Shift our array up one place and add the new price at $value\n";
        array_shift($icxtrend);
        array_push($icxtrend, $value);
        print "Calculate the moving average rounded to 8 decimal places\n";
        $movingavg = round((array_sum($icxtrend)/200),8);
        if($value >= $movingavg)
        {
          print "The price $value is equal to or above the moving average price of $movingavg\n";
        }
        else
        {
          print "The price $value is below the moving average price of $movingavg\n";
        }
      }
    }
  }
  sleep(5);
}
?>

So that is an example of doing some logic using the concepts I have explained.

This small program will collect data from Binance every 5 seconds, push the data into the array, calculate the moving average and tell us if the current price is equal to or above the moving average, or is below the moving average.

Remember, 200 iterations once every 5 seconds is going to take around 17 minutes to collect data and push into our array, after that, we then start calculating the average and doing some basic logic.

So watch as the program runs with a nice cup of coffee, cause we are going to use these concepts in creating our bot to automatically trade based on logical decisions we make. However, as with any good programmer, we don’t let it loose immediately, we perform all the logic and only do pretend trades to see how the bot performs, then adjust to cater for the all “what if” scenarios that are going to come up.

All those “what if” logical decisions is what we call Artificial Intelligence. The program is making logical decisions based on the data we collect in a manner that suits what we want to achieve.

Edit: press “CTRL-C” to stop the program at any time.

Stay Fishy


#187

Thanks for the tutorial Mike, very easy to follow. Looking forward to the next few lessons.

image


#188

Good stuff,

You might have noticed it got to iteration 199, which is actually 200 prices loaded into our array, as computers start counting from 0. Get use to the fact that zero is a number and a position in our array and zero is the first position. So zero is the first item, 1 is the second item, 2 is the third item and so on…

It’s a tricky one that catches many people new to programming.

Ultimately, we are going to create a bot similar to mine with less logic to keep it simple and yet it should still perform equivalent to this:

bot-1

My bot starts with 25 BTC and after 17,483 minutes, has had a return of 229.019%
IE: it has made 57.25 BTC in 291 hours, bringing my total to 82.25 BTC
For those who have watched the market over the last few days, which way has the price gone?
So how did the bot make so much?
All this is to come, where I will explain that there is no red, only green.

Binance has some minimum trade amounts, but they are quite small, we should be able to create an effective bot that will start trading with as little as $20. The more you have, the more trades you can perform at the same time and thus the more you can make and it compounds (as it makes more, it can use that additional amount to do more trades). Then there is the risk factor, just how much do you want to keep on the Exchange, which could all be lost.

You can sign up for Binance with the 2 BTC a day deposit and withdrawal limit without having to go through the KYC personal details stuff. 2 BTC a day is probably a lot for many people, but a bot can do more than this on average once it has built up a good pool to use.

You know, I only started with 1 BTC in Feb this year, now have a few hundred and you can scroll back in this thread to see all the things I did. I have learned so much and much of the information is in this thread. If you have time, browse through it as we will be coding our new little demo bot to do this type of stuff and nothing stopping you from doing the same as me if you put in the effort.

Give a man a Fish and he can eat for a day, teach a man to Fish and he can eat every day.

Stay Fishy


#189

I was scratching my head for a few minutes wondering how to get that code to run. Figured it out in the end, duh… used Notepad to create new text file and saved as a .php file.


#190

Quicker way, in your Powershell, type “notepad icx.php”
Notepad will open and ask you to create the new file, select Yes, then paste in code, save and exit.

Cheers…

Mike Fishy


#191

WOW. Thanks so much for this, Mike. I’ll be going through this over the weekend and giving it a whirl.


#192

Last post for today.

There is no red, only green.

nored-onlygreen

:joy:

If we look at Binance for the ICXUSDT pair, we see something like:

Notice the Yellow and Red lines, where they cross is where we want the bot to do potential trades to switch from holding ICX to holding USDT and vice versa.

So, when the red line crosses to be below the yellow line, we sell ICX to USDT.
So now we are holding USDT, so while it trends down, USDT is actually going up, it is green.
Then, when the red line crosses to be above the yellow line, we sell the USDT to buy ICX.
So now we are holding ICX, so while it trends up, ICX is going up against USDT and thus is green.

By switching between the two, we are effectively always green, though technically when we sell ICX we are shorting ICX and when we buy ICX, we are going Long on ICX.

In reality, we need to make some decisions around when to switch as well and we will build this in to cater for our decisions under the concept of the “what if” questions we are going to ask to the data we have.

So next will be a basic bot that will pretend to trade and do this switching without thought of the ramifications of doing the switching. It will be basic and not take into account many factors, but shows how we effectively run the strategy and we can run the program for a while to see how it does at this strategy (where it makes money and where it looses). When we see it in action, those “what if” questions start appearing, so we can see logically what to do when something happens, as it happens and make decisions in real time.

Stay Fishy


#193

FYI: later you can use this software for editing

Btw this is the topic I was waiting for a long time!


#194

@Mike_Fishy Thanks for the support, For my error Instead of copy paste the changed area, I manually edited the ini file and after that it worked. Great and Very interesting.


#195

Hi all,

OK, this next one is both a tutorial and a challenge.

Look at my post with the No Red, Only Green and the chart with the Yellow and Red Lines.

The Red Line is a short term moving average, while the Yellow Line is a longer term moving average.
As they cross, we want to perform a trade as things trend either up or down.

The code will assume the current value (price) is what we manage to buy or sell at, in reality this is not the case, but for this tutorial and challenge, we are going to assume the current price is what we manage get at the time the trade is performed. The goal of the challenge is to introduce you to why you need to worry about this and why you need to change those values on a semi-regular basis depending on what the market is doing. This is why so many bots you buy don’t always work, you need to adjust the strategy as conditions in the market changes. So a bot is not a set and forget thing, it needs updates, plus as Binance changes the API, the code also needs to be updated. This is why you need to learn how to code it yourselves, so you can maintain your code as things change. This is where your time will make you money.

So in the next bit of code I wrote, the bot does simulated trading and logs it to a file.

So, near the top of the code is the following:

$short_trend = 60;
print "Our short term trend line is $short_trend\n";
$long_trend = 100;
print "Our long term trend line is $long_trend\n";

This is the number of iterations through the loop for the long and short trend calculations.

Each iteration is 5 seconds, so 60 iterations is 5 minutes, while 100 iterations is 8.33 minutes.
I chose these as arbitrary values, your challenge is to change them to maximise your profits.

You can open 4 Powershell windows and run 4 of these at a time with different values.
Just save the code as a different filename for each one.
Also make sure to change the logfile name in the code for each one.

$tradefile = "ICX.txt";

So for each one, change the Logfile to something meaningful for you as this will log all the assumed buys and sells and the percentages for the program instance. Don’t try and run more than 4, it will time skew which is not detrimental, but not good either.

Binance charges 0.1% for each trade, so the code is subtracting 0.2% from your traded percentage after a completed Sell and Buy of ICX to cover the fees, this is to make it more realistic and fair.

The Short and Long trends are going to affect how aggressive or conservative the bot is.
The smaller the numbers, the more aggressive it will be.
The larger the numbers, the more conservative it will be.
The smaller the difference between the two numbers, the more aggressive it will be.
The larger the difference between the two numbers, the more conservative it will be.

So there is 4 things to consider when you play with those numbers.

Just like before, the bot will load price data into arrays, calculate moving averages and then has logic to determine when to buy and sell based on the moving averages crossing. As long as there is volatility in the market, the bot does well, but if things are flat, it will likely loose money as we are not taking this into account, plus several other things it is not taking into account that you will learn about by running it and playing with those numbers.

I tested the code for only a short period which did the following:

====================================
Iteration = 371
Running Time: 31.92 mins
Current running percentage = 0.119
====================================
Holding USDT as it trends up against ICX
Current ICX price is 0.23200000

The Loop can run for a month, so remember to press “CTRL-C” to stop the program at any time.

Check your logfile as well:

==========================
Runtime 19.1
Buy USDT at 0.23070000
==========================
==========================
Runtime 24.1
Sell USDT at 0.23160000
==========================
==========================
Runtime 24.1
TPC = 0.39 : RPC = 0.119 : Trade Completed

In my test case above, after the ICX was sold to USDT, then USDT was sold to ICX, there was a small gain even after the 0.2% was subtracted. So even my arbitrary numbers I choose in the code will work to some degree.

So, get this running, create 4 of them with different log files, set different numbers for the short and long term trend and start logging those trades and see how well or badly you go :slight_smile:

Fell free to post your numbers and results to the thread, gives us an idea of how long you ran it for and what was your percentage.

Code is in the next post to make it easier to copy/paste.

Stay Fishy


#196

Edit: Ignore this one, it wasn’t very good, use the next one.

Stay Fishy


#197

Version 2 of the PHP code.
Ignore that previous one, I removed it, it wasn’t good enough.

The new one makes more logical sense and provides more feedback to us as it runs.

As ICX is going up against USDT, we are not making more ICX, it’s just going up in value.

However, when ICX is going down against USDT and we buy USDT, we want more information, as this is the crucial time where we are going to try and make more ICX.

I have already seen a couple of “what if” things, but the objective is to let you see them.

Code:

<?php
require 'vendor/autoload.php';

$short_trend = 81;
print "Our short term trend line is $short_trend\n";
$long_trend = 151;
print "Our long term trend line is $long_trend\n";

$i = 0;
$binance_prices = array();
$icx_shorttrend = array();
$icx_longtrend = array();
$icx_shortavg = 0;
$icx_longavg = 0;
$time_start = time();
$time_end = 0;
$run_time = 0;
$tradefile = "ICX1.txt";
$icx = 1;
$buyvalue = 0;
$sellvalue = 0;
$rpc = 0;
$tpc = 0;

function getprices()
{
  $api = new Binance\API("<api key>","<secret>");
  $mp = $api->prices();
  return $mp;
}

for($i = 0; $i <= 2000000; $i++)
{
  $time_end = time();
  $run_time = round((($time_end - $time_start)/60),2);
  print "====================================\n";
  print "Iteration = $i \n";
  print "Running Time: $run_time mins \n";
  print "Current running percentage = $rpc \n";
  print "====================================\n";
  $binance_prices = getprices();
  foreach($binance_prices as $key => $value)
  {
    if($key == "ICXUSDT")
    {
      if($i < $short_trend)
      {
        array_push($icx_shorttrend, $value);
      }
      else
      {
        array_shift($icx_shorttrend);
        array_push($icx_shorttrend, $value);
      }
      if($i < $long_trend)
      {
        array_push($icx_longtrend, $value);
        print "Loading arrays with the price data\n";
      }
      else
      {
        array_shift($icx_longtrend);
        array_push($icx_longtrend, $value);
        $icx_shortavg = round((array_sum($icx_shorttrend)/$short_trend),8);
        $icx_longavg = round((array_sum($icx_longtrend)/$long_trend),8);
        if($icx_shortavg > $icx_longavg)
        {
          if($icx == 0)
          {
            print "Buying ICX with USDT at $value\n";
            $fh = fopen($tradefile, "a") or die("Cannot open file");
            fwrite($fh, "========================== \n");
            fwrite($fh, "Runtime $run_time \n");
            fwrite($fh, "Buy ICX at $value \n");
            fwrite($fh, "========================== \n");
            fclose($fh);
            $buyvalue = $value;
            $icx = 1;
          }
          print "Holding ICX as it trends up against USDT\n";
          print "Current ICX price is   : $value \n";
          print "Short Moving Average is: $icx_shortavg\n";
          print "Long Moving average is : $icx_longavg\n";
        }
        if($icx_shortavg < $icx_longavg)
        {
          if($icx == 1)
          {
            print "Selling ICX for USDT at $value\n";
            $fh = fopen($tradefile, "a") or die("Cannot open file");
            fwrite($fh, "========================== \n");
            fwrite($fh, "Runtime $run_time \n");
            fwrite($fh, "Sell ICX at $value \n");
            fwrite($fh, "========================== \n");
            fclose($fh);
            $sellvalue = $value;
            $icx = 0;
          }
          $ctpc = round(((($sellvalue - $value)/$value)*100),3);
          print "Holding USDT as it trends up against ICX\n";
          print "Current ICX price is   : $value \n";
          print "Sold ICX at            : $sellvalue\n";
          print "Current Trade Percent  : $ctpc\n";
          print "Short Moving Average is: $icx_shortavg\n";
          print "Long Moving average is : $icx_longavg\n";
        }
        if($icx == 1)
        {
          if($buyvalue != 0 AND $sellvalue != 0)
          {
            $tpc = round(((($sellvalue - $buyvalue)/$buyvalue)*100),3);
            $rpc = $rpc + ($tpc - 0.2);
            $fh = fopen($tradefile, "a") or die("Cannot open file");
            fwrite($fh, "========================== \n");
            fwrite($fh, "Runtime $run_time \n");
            fwrite($fh, "TPC = $tpc : RPC = $rpc : Trade Completed\n");
            fwrite($fh, "========================== \n");
            fclose($fh);
            $buyvalue = 0;
            $sellvalue = 0;
          }
        }
      }
    }
  }
  sleep(5);
}
?>

Provides better feedback and is more logical, concepts are the same.

Use this one and see if you can make it perform.

You can also watch the Binance charts and set the moving average lines to see visually what the bot is making decisions on. Here is an example of the bad bot trade:

Bad-Bot

You can see where the lines cross, so the bot sold ICX and then bought ICX, but the sell was lower than the buy and thus results in a loss. So one of the “what if” questions is: What if the long term trend is trending upwards? So not only do the lines need to cross, but they also need to both be trending in the same direction. In a later example, I will add more to the code to use an indicator to gives us a better idea of when to do the switching.

Stay Fishy


#198

Interesting they are all performing slightly differently, TBH, I don’t know which ones are which while it’s running. I’ll look into the log file later.


#199

Nice, I updated the code a little while ago, check you are running the one above.

2.4% an hour on that last one, it’s doing nicely, that’s like 57% a day.
If you were playing with $1000, that’s $570 a day.

Obviously amounts will vary, but can give you an idea of what you could achieve.

Also on a positive note, ICX is up 13% today when I last checked.

Stay Fishy


#200

The better performing one had the following settings:

$short_trend = 90;
$long_trend = 160;


#201

Current running percentage is the amount of extra ICX the bot has gained on the down swings, so you will actually gain more if the chart is trending down?

I was trying to follow the trades on Binance which seems to have gaps in the prices which may affect the bot with shorter moving averages.