Mike Fishy and Automation System Programming (Bots)


#404

In the next few days I will try to dig out some PHP code for pulling down the trading volumes unless someone already has the routines to hand.


#405

I have to try this after the exams :wink:


#406

Awesome Maclovin,

Codeacademy is pretty handy… suggest if you want help on coding its not bad.

Pay after 7 days.

Crash


#407

Actually we already have several more sample scripts for order depth and previous days close etc. Take a look in the examples directory from the php-binance-api GitHub repository which @Mike_Fishy got us to install.

Direct GitHub link is https://github.com/jaggedsoft/php-binance-api/tree/master/examples but the files are on your machine if you followed the original installation instructions.


#408

I don’t quite follow how Order Book Depth is a good metric for Volume of trades happening. I reckon that if tally the first 50 or 100 rows of the order book each loop, you can get a sense of how active traders are in a relative sense. Then there’s the matter of iceberg orders which essentially hide true order sizes and that alone can skew the volume metric quite a bit.

Seems to me if we want Volume data, we should be hitting aggregated trades or the clines API endpoints, or better yet plugging into either with websockets and tallying as the data flows through.


#409

I remember starting off by running my bots on a desktop, but eventually moved to a secure remote server so I can worry less about the physical hardware and focus on the strategies.

Nevertheless, this setup looks lovely :slight_smile:


#410

@rjbernaldo

I think the trick is to start to collect large datasets and to be able to look at them from a historical basis and to then pick indicators appropriately so that you can see what adjustments work the best. We have the structure of the bot more or less, Now what would be good is to be able to model performance from a historical basis.

This is why I am concetrating on pulling 5 months of data next on a datacentre PC. You are going to come up with a massive dataset so that you can start to model the performance of you bot over a long time frame.

If you have collectd the data there is nothing stoping you of running tests on it (Replay) on six months to effectively learn how to pick up the ups and down.

We are in a place of being able to store the data so we can model performance.

Hopefully this makes sense.

Crash


#411

@Mike_Fishy, I thought I’d address Redis vs. MongoDB a little further after giving it some thought…

For the amount of data we’re talking about collecting and storing, Redis is not really the best choice and certainly not a choice for any sort of long-term/historical holder of data. But that doesn’t mean it doesn’t have a place, so I thought I’d share a bit more about what I do with Redis.

For me and how I use Redis, it makes it possible to write bots in Ruby without having to resort to a compilable language. I can spin up many Ruby threads, one per market, or one per websocket connection and so on and turn the data into Ruby objects and serialize them to the Redis cache and that data becomes nearly instantly available to the bots doing the analyzing and buy/sell action. I do this to maintain “live” Tickers, Order books, last 3000 trades, last 3000 clines (1 minute candles up to daily candles) and buy/sell orders as they fill.

The websockets feed the Redis cache as the data comes in and so I code the rest of the system to expect the Redis caches to be latest data available from the Binance server. There’s a timestamp on everything in Redis, so these bots can fallback to RESTful API’s fairly seamlessly and keep churning, or at least start winding down out of open positions.

So Redis’ main role for me is storing serialized Ruby objects that already have CPU intensive computations done (i.e. for clines, several indicators computed like RSI, ATR, EMA, etc.). The other big role for Redis is opening up the data door, so to speak, between Ruby threads so data can be produced and cached, yet seamlessly accessed from any other thread in parallel without having to write ironclad thread safe code.

Having said all that, the way I see it, and depending on whether a fully functioning and trading bot is built by this community, there are three ways forward:

  1. Build exclusively with MongoDB – probably simplest way forward and fewer moving parts. But I think performance-wise, down the road, one has to start thinking of going to a compilable language to handle the number of market pairs out there efficiently, esp. in a loop like we’re doing here so far.

  2. Build with Redis and hold short windowed amounts of data in RAM and have another process that plucks and stores long-term (MongoDB or Postgres or MySQL).

  3. Third option, not yet considered: Postgres or MySQL and use the NoSQL facilities of these. I don’t know about you guys, but I store every trade and data about the opening and closing performance of those trades in a database, so my project already involves MySQL + Redis so I’m not all that inclined to build in MongoDB and have three storage facilities to maintain. However, I’m not disinclined to simply start storing more data longer with a NoSQL approach like MongoDB and can see many benefits to storing reams of JSON data untouched as delivered via the API’s straight to the DB for later consumption/replay options.

For those of you trying to get your head around this discussion, Panopoly has a great set of comparison articles that can get you up to speed on the jargon and what’s what of these tools:



The difference between SQL and NoSQL (i.e. MongoDB):


#412

@mwlang Redis is good for it’s purpose, but when you program on Linux, you can use arrays in shared memory and it does the same thing. All the processes can access the shared regions and thus Redis is sort of included in your programs.

I have not been coding at the moment, I’m building a new AMD Ryzen system :smiley:

Will contribute some more once I got this beast sorted.

To be honest though, for the platform, Windows is for desktops and Linux is for servers. It makes sense for it to run on Linux, but using PHP to prototype ideas can run on both.

If you install Ubuntu Linux, to setup MongoDB, it is just “sudo apt install mongodb” and the system does the rest. One of the advantages of Linux, all the software is there and mostly free. If you need instructions for installing on Windows or any other software installs, let me know, I have done pretty much all of them.

For the Orders, when we get to that part, you want to have that cycle well known by the bot. When the bot places and order, if it doesn’t fill in a short time frame, you want to cancel it and wait for the next cycle - it frees up funds to use elsewhere. Also, placing limit orders can help the bot be more profitable, as the limit order is often less than the current value - the really hard goal is to get that limit order in when the indications are right and before it starts going up - which needs some predictive maths :slight_smile:

One of the advantages of PHP, the maths is already done for you.
https://php-ml.readthedocs.io/en/latest/machine-learning/classification/k-nearest-neighbors/

Stay Fishy


#413

A copy of the latest script provided by @mwlang but with stop loss included (and using the new notation of -> not .).

With Notepad++ rather than Notepad you get the Linux file ‘tail’ effect where you can watch data being written to the end of the logging files. However I noticed that even though the files are set as append mode (a) the script is currently recreating the data file each time you run the script. My plan is just to let the data log get as big as possible for back-testing purposes.

I know a very basic txt file for a few hours / days isn’t an ideal solution but it’s better than nothing for now. Let me know if anyone can see why the data is not being appended to the API data log.

Until we are all up to speed with GitHub I refer to this file as V0383a.php as an amendment to the script in post # 0383. But if you use it, without modification, it will be V0413.php.

<?php  // V0383a.php https://thebitcoin.pub/t/mike-fishy-and-automation-system-programming-bots/36481/383
require 'vendor/autoload.php';

// This bot will trade USDT to the other pairs and back to make more USDT
// It shall use all the trading pairs to make more UDST except the ones we tell it not to use

$interval = 5;            // interval between price sampling
$stperiod = 50;           // Short Term Trend - must be less than $mtperiod
$mtperiod = 90;           // Medium Term Trend - must be less than $ltperiod
$ltperiod = 120;          // Long Term Trend
$tradefile = "BTC.txt";   // The Trade Logging file name
$minspread = 1.1;         // The minimum spread percentage needed for a trade
$minrsi = 45;             // Relative Strength must be below this number to buy
$sellbuffer = 1.003;      // Create a buffer to hold CDA if sell is not profitable
$maxorders = 10;          // Maximum number of concurrent orders
$stoplossbuffer = 0.995;   // Stop Loss buffer to sell CDA if it is forming a loss (0.5% Stop Loss threshold)  //rev a

// Do not change any of the flags, we use this to signal the bot what to do and when
$buyready = 0;            // This flag signals the bot that the pair meets rules to buy
$buyprep = 1;             // This flag signals the bot to prepare to buy
$buyord = 2;              // This flag signals the bot to place an order
$sellok = 3;              // This flag signals the bot that the order was completed
$sellready = 4;           // This flag signals the bot to sell
$selldone = 5;            // This flag signals the bot the trade completed
$dontbuy = 6;             // This flag signals the bot we dont want to trade BCASH :P

// Trend Directions
$unset = 0;               // Initial trend is indeterminent until set otherwise.
$down = 1;                // price action is trending downward
$flat = 2;                // price action is flat/unchanged
$up = 3;                  // price action is trending upward

// Standard variables and arrays we use
$replay = 0;
$i = 0;
$cdas = array();
$binance_prices = array();
$time_start = time();
$time_end = 0;
$run_time = 0;
$rpc = 0;
$tpc = 0;
$q = 0;
$cdaorders = 0;
$btcprice = 0;
$btctrend = array();
$bttrend = "WAIT";

class Trend {
  public $period = 0;
  public $label = '';
  public $values = array();
  public $direction = 0;
  public $rsi = 100;
  public $value = 0;
  public $prev_value = 0;
  protected $rsi_gains = array();
  protected $rsi_losses = array();

  function __construct ($label, $period) {
    $this->label = $label;
    $this->period = $period;
  }

  public function stat () {
    global $down;
    global $flat;
    global $up;

    print "\t $this->label:";
    printf("%-14.8F",$this->avg());
    if ($this->direction == $down) printf("%-5s",":DOWN");
    if ($this->direction == $flat) printf("%-5s",":FLAT");
    if ($this->direction == $up) printf("%-5s",":UP");
  }

  // Calculate the Relative Strength Indicator on the Array
  // A Low RSI indicates a buy opportunity
  function compute_rsi () {

    if (empty($this->rsi_gains) OR $this->value == $this->prev_value) {
      array_push($this->rsi_gains, 0);
      array_push($this->rsi_losses, 0);
    
    } elseif ($this->value > $this->prev_value) {
      array_push($this->rsi_gains, $this->value - $this->prev_value);
      array_push($this->rsi_losses, 0);

    } else {
      array_push($this->rsi_losses, $this->prev_value - $this->value);
      array_push($this->rsi_gains, 0);
    }

    if (count($this->rsi_gains) > $this->period) {
      array_shift($this->rsi_gains);
      array_shift($this->rsi_losses);
    }

    $gain_avg = array_sum($this->rsi_gains) / count($this->rsi_gains);
    $loss_avg = array_sum($this->rsi_losses) / count($this->rsi_gains);

    if ($loss_avg > 0) {
      $this->rsi = round(100-(100/(1+($gain_avg/$loss_avg))),3);

    } else {
      $this->rsi = 100;
    }
  }

  public function append ($value) {
    $this->prev_value = end($this->values);
    $this->value = $value;

    array_push($this->values, $value);
    if (count($this->values) > $this->period) array_shift($this->values);
    $this->compute_rsi();
  }

  public function avg () { 
    if (count($this->values) == 0) {
      return 0;
    } else {
      return round((array_sum($this->values) / count($this->values)), 8);
    }
  }

  // Calculate the spread, which is the percentage difference between
  // the highest recorded price and the lowest recorded price in the Trend Array
  public function spread () {
    $low = min($this->values);
    $high = max($this->values);
    return round(((1 - ($low/$high)) * 100), 3);
  }

  public function set_direction ($other) {
    global $down;
    global $flat;
    global $up;

    $avg = $this->avg();
    if ($other < $avg) $this->direction = $down;
    if ($other == $avg) $this->direction = $flat;
    if ($other > $avg) $this->direction = $up;
  }
}

class Cda {

  public $tick = '';                // The ticker symbol for this CDA
  public $st = NULL;                // Short Term Trend
  public $mt = NULL;                // Medium Term Trend
  public $lt = NULL;                // Long Term Trend
  public $tradeflag = 0;            // Set this pair to buyready
  public $buyvalue = 0;             // record what we buy for on this pair
  public $sellvalue = 0;            // record what we sell for on this pair
  public $lasttrade = 0;            // record we have had one trade done
  public $lasttpc = 0;              // record what percentage last the trade was
  public $isset = 1;                // used to signal we are initialised for this pair
  public $trade_start = 0;          // start of currently open trade
  public $trade_cycles = array();   // track each trade cycle length

  function __construct($tick) {
    global $stperiod;
    global $mtperiod;
    global $ltperiod;

    $this->tick = $tick;
    $this->st = new Trend('ST', $stperiod);
    $this->mt = new Trend('MT', $mtperiod);
    $this->lt = new Trend('LT', $ltperiod); 
  }

  function dontbuy() {
    global $dontbuy;
    $this->tradeflag = $dontbuy;
  }

  function update_price($value) {
    $this->st->append($value);
    $this->mt->append($value);
    $this->lt->append($value);
  }
}

// API call to fetch pricing data from Binance
function getprices()
{
  global $replay;
  global $pricefh;
  
  if($replay == 0)
  {
    $api = new Binance\API("<api key>","<secret>");
    $mp = $api->prices();
    $pricefh = fopen("BTC-prices.txt","a") or die("Cannot open file");
    fwrite($pricefh, serialize($mp) . "\n");
    fclose($pricefh);
  }
  else
  {
    if (!$pricefh)
    {
      $pricefh = fopen("BTC-prices.txt","r") or die("Cannot open replay file");
    }
    $mp = unserialize(fgets($pricefh));
    if ($mp == NULL) die("end of price data reached.\n");
  }
  return $mp;
}

if ($replay == 1) 
{
  $simulation_mode = "replaying"; 
}
else 
{
  $simulation_mode = "real-time";
  if(file_exists("BTC-prices.txt"))
  {
    unlink("BTC-prices.txt");
  }
}

// Start of the Loop - can run for months - press CTRL-C to stop
for($i = 0; $i <= 2000000; $i++)
{
  $time_end = time();
  if ($replay == 1) {
    $run_time = round(($i * $interval) / 60, 2);
  } else {
    $run_time = round((($time_end - $time_start)/60),2);
  }
  print "====================================\n";
  print "Iteration = $i ($simulation_mode) \n";
  print "Running Time: $run_time mins \n";
  print "Current BTC Price is: $btcprice T:$bttrend \n";
  print "Current Orders in progress = $cdaorders / $maxorders\n";
  print "Current running percentage = $rpc \n";
  print "====================================\n";

  // Fetch current prices from Binance
  $binance_prices = getprices();

  // Loop through the price data as key and value pairs
  foreach($binance_prices as $key => $value)
  {
    // Track BTC price for display
    if($key == "BTCUSDT")
    {
      $btcprice = $value;
      array_push($btctrend, $value);
      if($i >= $cda->mt->period)
      {
        array_shift($btctrend);
        $btcavg = round((array_sum($btctrend)/$cda->lt->period),8);
        if($value >= $btcavg)
        {
          $bttrend = "UP";
        }
        else
        {
          $bttrend = "DOWN";
        }
      }
    }
    // Only process pairs with BTC
    if(strpos($key, "BTC"))
    {

      // Convert the pair name to lower case in varibale $tick
      // for example the name "BTCUSDT" will become "btcusdt"

      // Use the lower case name to form the leading part of varibales and arrays
      // for exmaple, using "btcusdt" and adding "st" for the short term array
      // will initialise an array called "btcusdtst"
      // as we loop thorugh the pairs, each one gets created for each pair
      // for exmaple "NEOUSDT" will become "neousdtst"
      $tick = strtolower($key);

      // Check if the trading pair has been initialised
      // this covers if Binance add a new trading pair on USDT while we are running
      // if Binance adds new trading pairs while bot is running, we shall
      // ignore them and only use the ones since the bot was started and initialised
      if (!isset($cdas[$tick])) $cdas[$tick] = new Cda($tick);

      $cda = $cdas[$tick];

      // Exclude List - these ones we do not trade

      if($key == "BCHABCBTC") $cda->dontbuy();
      if($key == "BCHSVBTC") $cda->dontbuy();
      if($key == "BCCBTC") $cda->dontbuy();

      // Push data into arrays and shift arrays once we have enough data
      $cda->update_price($value);
      $cda->st->set_direction($value);
      $cda->mt->set_direction($cda->st->avg());
      $cda->lt->set_direction($cda->mt->avg());

      // Wait until we have all the arrays populated with data
      if($i <= $cda->lt->period) {
        if($key == "BCHSVBTC") print "Loading Arrays with data until Iteration " . $cda->lt->period . " - patience, Grasshopper...\n";
      }

      // Arrays are populated, so on with the processing
      else
      {

        // Print out only ones in Buy Order and Sell Ready
        if($cda->tradeflag == $buyord OR $cda->tradeflag == $sellready) 
        {
          printf("%-9s",$key);
          print "\tV:";
          printf("%-14.8F",$value);

          if($cda->tradeflag == $sellready) {
            print "\tCTC:";
            printf("%-3.2F", ($run_time - $cda->trade_start));
          } else {
            print "\tATC:";
            printf("%-3.2F", $cda->trade_cycle_avg);
          }

          print "\t" . $cda->st->stat();
          print "\t" . $cda->mt->stat();
          print "\t" . $cda->lt->stat();

          print "\t  SPREAD:";
          printf("%-03.3F",$cda->lt->spread()); 
          print "%\t  RSI:";
          printf("%-06.3F",$cda->st->rsi);
          printf("/%-06.3F",$cda->mt->rsi);
          printf("/%-06.3F",$cda->lt->rsi);

          if($cda->tradeflag == $buyord) $cdastatus = "Buy Order";
          if($cda->tradeflag == $sellready) $cdastatus = "Sell Ready";
          if($cda->tradeflag == $sellready)
          {
            $ctp = round(((($value - $cda->buyvalue)/$cda->buyvalue)*100),3);
            print "\t S:$cdastatus \tBV:$cda->buyvalue CTP:$ctp";
          }
          else
          {
            print "\tS:$cdastatus";
          }
          if($cda->lasttrade == 1)
          {
            print "   LastTPC:$cda->lasttpc";
          }
          print "\n";
        }

        // Trading rules start here
        // ========================

        // CDA is trending up so set to buyprep
        if($cda->tradeflag == $buyready AND $cda->st->direction==$up AND $cda->mt->direction==$up AND $cda->lt->direction==$up)
        {
          printf("%-9s",$key);
          print "Was Buyready, now Buyprep V:$value\n";
          $cda->tradeflag = $buyprep;
        }

        // CDA was buyprep, now trending down, set to buyord if reasonable spread
        if($cda->tradeflag == $buyprep AND $cda->st->direction==$down AND $cda->mt->direction==$down AND $cda->lt->direction==$down AND $cda->lt->spread() >= $minspread)
        {
          printf("%-9s",$key);
          print "Was Buyprep, now Buyord V:$value\n";
          $cda->tradeflag = $buyord;
        }

        // CDA stopped trending down and is ready to buy
        if($cda->tradeflag == $buyord AND $cda->st->direction==$up AND $cda->mt->direction!=$down)
        {
          if($cda->lt->rsi <= $minrsi)
          {
            if($cdaorders < $maxorders)
            {
              printf("%-9s",$key);
              print "Was Buyord, now Buy V:$value\n";
              // Assume we buy at the current value
              $cda->buyvalue = $value;
              $cda->tradeflag = $sellok;
              $cda->trade_start = $run_time;
              $cdaorders = $cdaorders + 1;
              $fh = fopen($tradefile, "a") or die("Cannot open file");
              fwrite($fh, "========================== \n");
              fwrite($fh, "Runtime $run_time \n");
              fwrite($fh, "Buy on $key BV:$cda->buyvalue \n");
              fwrite($fh, "========================== \n");
              fclose($fh);
            }
          }
          else
          {
            printf("%-9s",$key);
            print "RSI Check not meeting Minimum, resetting back to Buy Prep\n";
            $cda->tradeflag = $buyprep;
          }
        }

        // Buy Order on CDA placed, do order tracking here to make sure order completes
        if($cda->tradeflag == $sellok)
        {
          // Since we are not placing an order, we just assume it completed
          $cda->tradeflag = $sellready;
        }

        // CDA is sellready and is no longer trending upwards - time to sell
        if($cda->tradeflag == $sellready AND $cda->st->direction!=$up AND $cda->mt->direction!=$up)
        {
          // Assume we sell at the current value and not sell if not meeting a minimum
          $cdabuff = $cda->buyvalue * $sellbuffer;
          if($value > $cdabuff)
          {
            $cda->sellvalue = $value;
            $cda->tradeflag = $selldone;
          }
          else
          {
            printf("%-9s",$key);
            print "Did not meet minimum sell amount\n";
          }
        }
		
		// Stop Loss all added as part of my rev a but don't use the old $tick."buyvalue" motation
		//$cdastoploss = ${$tick . "buyvalue"} * $stoplossbuffer;
		$cdastoploss = $cda->buyvalue * $stoplossbuffer;
		//if(${$tick . "buyflag"} == $sellready AND ($value < $cdastoploss))
		if($cda->buyvalue == $sellready AND ($value < $cdastoploss))
		{
			printf("%-9s",$key);
			print "hit the stop loss buffer, so we are selling out\n";
			${$tick . "sellvalue"} = $value;
			${$tick . "buyflag"} = $selldone;							
		}		

        // CDA is selldone
        if($cda->tradeflag == $selldone)
        {
          // Sell Order on CDA placed, do order tracking here to make sure order completes
          // Since we are not placing an order, we just assume it completed
          $q = round(((($cda->sellvalue - $cda->buyvalue)/$cda->buyvalue)*100),3);
          $tpc = $q - 0.2;
          $rpc = round($rpc + ($tpc/$maxorders),3);
          $cda->lasttrade = 1;
          $cda->lasttpc = $tpc;
          $cda->tradeflag = $buyready;
          array_push($cda->trade_cycles, ($run_time - $cda->trade_start)); 
          $cda->trade_cycle_avg = round((array_sum($cda->trade_cycles)/count($cda->trade_cycles)),3);
          $cdaorders = $cdaorders - 1;
          printf("%-9s",$key);
          print "Sell Done BV:$cda->buyvalue SV:$cda->sellvalue TPC:$tpc ATC:$cda->trade_cycle_avg\n";
          $fh = fopen($tradefile, "a") or die("Cannot open file");
          fwrite($fh, "========================== \n");
          fwrite($fh, "Runtime $run_time \n");
          fwrite($fh, "Sell Done on $key BV:$cda->buyvalue SV:$cda->sellvalue TPC:$tpc RPC:$rpc ATC:$cda->trade_cycle_avg\n");
          fwrite($fh, "========================== \n");
          fclose($fh);
        }

      }
    }
  }
  if ($replay == 0) sleep($interval);
}
?>

Edit: line 321 is currently coming up with an undefined property error but the script seems to be running ok.

printf("%-3.2F", $cda->trade_cycle_avg);

#414

Add below line 56 the following public property declaration of the CDA class to fix the undefined error:

  public $trade_cycle_avg = 0;

#415

@mwlang bug still present even after the new line 57.


#416

Are you talking about PHP’s http://php.net/manual/en/function.shmop-open.php ?

Ruby doesn’t normally expose shared memory like this. Shared memory can be utilized and accessed correctly (i.e. thread-safe) if you code threds correctly and use Semaphores, etc. and synchronize before accessing shared memory. OS doesn’t matter here.

require 'thread'
semaphore = Mutex.new

a = Thread.new {
  semaphore.synchronize {
    # access shared resource
  }
}

b = Thread.new {
  semaphore.synchronize {
    # access shared resource
  }
}

All I was saying was that Redis allowed me to do away with Thread safety concerns of the language and just build it without thinking about synchronizing with mutexes and semaphores.

Regardless, you’re right that mongodb is dirt simple to install and start using, esp. on Linux systems.


#417

Being able to test your strategy on historical data is absolutely critical. It’s one of the key features I knew needed to be included in the initial release of https://coinfu.io

We must always remember the adage though that “past performance is not an indicator of future performance.” Never model your strategy too tightly according to the data. I’d like to think of backtests as more of a guide rather than the ultimate solution.


#418

Mike I should have a Linux system up by the end of the week. Presuming that to install on Windows it says you need a copy of windows server 2012 r2. This to me means the need to install VMware on my WIndows Machine and installing windows server 2012 r2.

Have i got this right? If you can install VM ware… you could also install Linux too.

Anyone got any thoughts on this?

Crash


#419

Fixed the code with the extract below. It can only show trade_cycle_avg after the first trade of the ticker has been completed. Without the first trade the array doesn’t exist so it does a check to see if trade_cycles > 0.

          if($cda->tradeflag == $sellready) {
            print "\tCTC:";                             // just tickers with sellready status
            printf("%-3.2F", ($run_time - $cda->trade_start));
          } else {                     					// just tickers with buyord status
            print "\tATC:";
			if(count($cda->trade_cycles) > 0)   		// can show trade_cycle_avg after first trade has completed
			{
				printf("%-3.2F", $cda->trade_cycle_avg);
			}
			else
			{
				print "--.--";                        	// no trades completed yet so no average available
			}
          }

I also fixed the stop loss code as I hadn’t changed all of it to the new notation. If you don’t want to run with a stop loss just set $stoplossbuffer at line 16 to 0.

<?php  // V0419.php was V0383a.php https://thebitcoin.pub/t/mike-fishy-and-automation-system-programming-bots/36481/383
require 'vendor/autoload.php';

// This bot will trade USDT to the other pairs and back to make more USDT
// It shall use all the trading pairs to make more UDST except the ones we tell it not to use

$interval = 5;            // interval between price sampling
$stperiod = 50;           // Short Term Trend - must be less than $mtperiod
$mtperiod = 90;           // Medium Term Trend - must be less than $ltperiod
$ltperiod = 120;          // Long Term Trend
$tradefile = "BTC.txt";   // The Trade Logging file name
$minspread = 1.1;         // The minimum spread percentage needed for a trade
$minrsi = 45;             // Relative Strength must be below this number to buy
$sellbuffer = 1.003;      // Create a buffer to hold CDA if sell is not profitable
$maxorders = 10;          // Maximum number of concurrent orders
$stoplossbuffer = 0.995;  // Stop Loss buffer to sell CDA if it is forming a loss (0.5% Stop Loss threshold), set to 0 to disable stop loss

// Do not change any of the flags, we use this to signal the bot what to do and when
$buyready = 0;            // This flag signals the bot that the pair meets rules to buy
$buyprep = 1;             // This flag signals the bot to prepare to buy
$buyord = 2;              // This flag signals the bot to place an order
$sellok = 3;              // This flag signals the bot that the order was completed
$sellready = 4;           // This flag signals the bot to sell
$selldone = 5;            // This flag signals the bot the trade completed
$dontbuy = 6;             // This flag signals the bot we dont want to trade BCASH :P

// Trend Directions
$unset = 0;               // Initial trend is indeterminent until set otherwise.
$down = 1;                // price action is trending downward
$flat = 2;                // price action is flat/unchanged
$up = 3;                  // price action is trending upward

// Standard variables and arrays we use
$replay = 0;
$i = 0;
$cdas = array();
$binance_prices = array();
$time_start = time();
$time_end = 0;
$run_time = 0;
$rpc = 0;
$tpc = 0;
$q = 0;
$cdaorders = 0;
$btcprice = 0;
$btctrend = array();
$bttrend = "WAIT";

class Trend {
  public $period = 0;
  public $label = '';
  public $values = array();
  public $direction = 0;
  public $rsi = 100;
  public $value = 0;
  public $prev_value = 0;
  public $trade_cycle_avg = 0;
  protected $rsi_gains = array();
  protected $rsi_losses = array();

  function __construct ($label, $period) {
    $this->label = $label;
    $this->period = $period;
  }

  public function stat () {
    global $down;
    global $flat;
    global $up;

    print "\t $this->label:";
    printf("%-14.8F",$this->avg());
    if ($this->direction == $down) printf("%-5s",":DOWN");
    if ($this->direction == $flat) printf("%-5s",":FLAT");
    if ($this->direction == $up) printf("%-5s",":UP");
  }

  // Calculate the Relative Strength Indicator on the Array
  // A Low RSI indicates a buy opportunity
  function compute_rsi () {

    if (empty($this->rsi_gains) OR $this->value == $this->prev_value) {
      array_push($this->rsi_gains, 0);
      array_push($this->rsi_losses, 0);
    
    } elseif ($this->value > $this->prev_value) {
      array_push($this->rsi_gains, $this->value - $this->prev_value);
      array_push($this->rsi_losses, 0);

    } else {
      array_push($this->rsi_losses, $this->prev_value - $this->value);
      array_push($this->rsi_gains, 0);
    }

    if (count($this->rsi_gains) > $this->period) {
      array_shift($this->rsi_gains);
      array_shift($this->rsi_losses);
    }

    $gain_avg = array_sum($this->rsi_gains) / count($this->rsi_gains);
    $loss_avg = array_sum($this->rsi_losses) / count($this->rsi_gains);

    if ($loss_avg > 0) {
      $this->rsi = round(100-(100/(1+($gain_avg/$loss_avg))),3);

    } else {
      $this->rsi = 100;
    }
  }

  public function append ($value) {
    $this->prev_value = end($this->values);
    $this->value = $value;

    array_push($this->values, $value);
    if (count($this->values) > $this->period) array_shift($this->values);
    $this->compute_rsi();
  }

  public function avg () { 
    if (count($this->values) == 0) {
      return 0;
    } else {
      return round((array_sum($this->values) / count($this->values)), 8);
    }
  }

  // Calculate the spread, which is the percentage difference between
  // the highest recorded price and the lowest recorded price in the Trend Array
  public function spread () {
    $low = min($this->values);
    $high = max($this->values);
    return round(((1 - ($low/$high)) * 100), 3);
  }

  public function set_direction ($other) {
    global $down;
    global $flat;
    global $up;

    $avg = $this->avg();
    if ($other < $avg) $this->direction = $down;
    if ($other == $avg) $this->direction = $flat;
    if ($other > $avg) $this->direction = $up;
  }
}

class Cda {

  public $tick = '';                // The ticker symbol for this CDA
  public $st = NULL;                // Short Term Trend
  public $mt = NULL;                // Medium Term Trend
  public $lt = NULL;                // Long Term Trend
  public $tradeflag = 0;            // Set this pair to buyready
  public $buyvalue = 0;             // record what we buy for on this pair
  public $sellvalue = 0;            // record what we sell for on this pair
  public $lasttrade = 0;            // record we have had one trade done
  public $lasttpc = 0;              // record what percentage last the trade was
  public $isset = 1;                // used to signal we are initialised for this pair
  public $trade_start = 0;          // start of currently open trade
  public $trade_cycles = array();   // track each trade cycle length

  function __construct($tick) {
    global $stperiod;
    global $mtperiod;
    global $ltperiod;

    $this->tick = $tick;
    $this->st = new Trend('ST', $stperiod);
    $this->mt = new Trend('MT', $mtperiod);
    $this->lt = new Trend('LT', $ltperiod); 
  }

  function dontbuy() {
    global $dontbuy;
    $this->tradeflag = $dontbuy;
  }

  function update_price($value) {
    $this->st->append($value);
    $this->mt->append($value);
    $this->lt->append($value);
  }
}

// API call to fetch pricing data from Binance
function getprices()
{
  global $replay;
  global $pricefh;
  
  if($replay == 0)
  {
    $api = new Binance\API("<api key>","<secret>");
    $mp = $api->prices();
    $pricefh = fopen("BTC-prices.txt","a") or die("Cannot open file");
    fwrite($pricefh, serialize($mp) . "\n");
    fclose($pricefh);
  }
  else
  {
    if (!$pricefh)
    {
      $pricefh = fopen("BTC-prices.txt","r") or die("Cannot open replay file");
    }
    $mp = unserialize(fgets($pricefh));
    if ($mp == NULL) die("end of price data reached.\n");
  }
  return $mp;
}

if ($replay == 1) 
{
  $simulation_mode = "replaying"; 
}
else 
{
  $simulation_mode = "real-time";
  if(file_exists("BTC-prices.txt"))
  {
    unlink("BTC-prices.txt");
  }
}

// Start of the Loop - can run for months - press CTRL-C to stop
for($i = 0; $i <= 2000000; $i++)
{
  $time_end = time();
  if ($replay == 1) {
    $run_time = round(($i * $interval) / 60, 2);
  } else {
    $run_time = round((($time_end - $time_start)/60),2);
  }
  print "====================================\n";
  print "Iteration = $i ($simulation_mode) \n";
  print "Running Time: $run_time mins \n";
  print "Current BTC Price is: $btcprice T:$bttrend \n";
  print "Current Orders in progress = $cdaorders / $maxorders\n";
  print "Current running percentage = $rpc \n";
  print "====================================\n";

  // Fetch current prices from Binance
  $binance_prices = getprices();

  // Loop through the price data as key and value pairs
  foreach($binance_prices as $key => $value)
  {
    // Track BTC price for display
    if($key == "BTCUSDT")
    {
      $btcprice = $value;
      array_push($btctrend, $value);
      if($i >= $cda->mt->period)
      {
        array_shift($btctrend);
        $btcavg = round((array_sum($btctrend)/$cda->lt->period),8);
        if($value >= $btcavg)
        {
          $bttrend = "UP";
        }
        else
        {
          $bttrend = "DOWN";
        }
      }
    }
    // Only process pairs with BTC
    if(strpos($key, "BTC"))
    {

      // Convert the pair name to lower case in varibale $tick
      // for example the name "BTCUSDT" will become "btcusdt"

      // Use the lower case name to form the leading part of varibales and arrays
      // for exmaple, using "btcusdt" and adding "st" for the short term array
      // will initialise an array called "btcusdtst"
      // as we loop thorugh the pairs, each one gets created for each pair
      // for exmaple "NEOUSDT" will become "neousdtst"
      $tick = strtolower($key);

      // Check if the trading pair has been initialised
      // this covers if Binance add a new trading pair on USDT while we are running
      // if Binance adds new trading pairs while bot is running, we shall
      // ignore them and only use the ones since the bot was started and initialised
      if (!isset($cdas[$tick])) $cdas[$tick] = new Cda($tick);

      $cda = $cdas[$tick];

      // Exclude List - these ones we do not trade

      if($key == "BCHABCBTC") $cda->dontbuy();
      if($key == "BCHSVBTC") $cda->dontbuy();
      if($key == "BCCBTC") $cda->dontbuy();

      // Push data into arrays and shift arrays once we have enough data
      $cda->update_price($value);
      $cda->st->set_direction($value);
      $cda->mt->set_direction($cda->st->avg());
      $cda->lt->set_direction($cda->mt->avg());

      // Wait until we have all the arrays populated with data
      if($i <= $cda->lt->period) {
        if($key == "BCHSVBTC") print "Loading Arrays with data until Iteration " . $cda->lt->period . " - patience, Grasshopper...\n";
      }

      // Arrays are populated, so on with the processing
      else
      {

        // Print out only ones in Buy Order and Sell Ready
        if($cda->tradeflag == $buyord OR $cda->tradeflag == $sellready) 
        {
          printf("%-9s",$key);
          print "\tV:";
          printf("%-14.8F",$value);

          if($cda->tradeflag == $sellready) {
            print "\tCTC:";                             // just tickers with sellready status
            printf("%-3.2F", ($run_time - $cda->trade_start));
          } else {                     					// just tickers with buyord status
            print "\tATC:";
			if(count($cda->trade_cycles) > 0)   		// can show trade_cycle_avg after first trade has completed
			{
				printf("%-3.2F", $cda->trade_cycle_avg);
			}
			else
			{
				print "--.--";                        	// no trades completed yet so no average available
			}
          }

          print "\t" . $cda->st->stat();
          print "\t" . $cda->mt->stat();
          print "\t" . $cda->lt->stat();

          print "\t  SPREAD:";
          printf("%-03.3F",$cda->lt->spread()); 
          print "%\t  RSI:";
          printf("%-06.3F",$cda->st->rsi);
          printf("/%-06.3F",$cda->mt->rsi);
          printf("/%-06.3F",$cda->lt->rsi);

          if($cda->tradeflag == $buyord) $cdastatus = "Buy Order";
          if($cda->tradeflag == $sellready) $cdastatus = "Sell Ready";
          if($cda->tradeflag == $sellready)
          {
            $ctp = round(((($value - $cda->buyvalue)/$cda->buyvalue)*100),3);
            print "\t S:$cdastatus \tBV:$cda->buyvalue CTP:$ctp";
          }
          else
          {
            print "\tS:$cdastatus";
          }
          if($cda->lasttrade == 1)
          {
            print "   LastTPC:$cda->lasttpc";
          }
          print "\n";
        }

        // Trading rules start here
        // ========================

        // CDA is trending up so set to buyprep
        if($cda->tradeflag == $buyready AND $cda->st->direction==$up AND $cda->mt->direction==$up AND $cda->lt->direction==$up)
        {
          printf("%-9s",$key);
          print "Was Buyready, now Buyprep V:$value\n";
          $cda->tradeflag = $buyprep;
        }

        // CDA was buyprep, now trending down, set to buyord if reasonable spread
        if($cda->tradeflag == $buyprep AND $cda->st->direction==$down AND $cda->mt->direction==$down AND $cda->lt->direction==$down AND $cda->lt->spread() >= $minspread)
        {
          printf("%-9s",$key);
          print "Was Buyprep, now Buyord V:$value\n";
          $cda->tradeflag = $buyord;
        }

        // CDA stopped trending down and is ready to buy
        if($cda->tradeflag == $buyord AND $cda->st->direction==$up AND $cda->mt->direction!=$down)
        {
          if($cda->lt->rsi <= $minrsi)
          {
            if($cdaorders < $maxorders)
            {
              printf("%-9s",$key);
              print "Was Buyord, now Buy V:$value\n";
              // Assume we buy at the current value
              $cda->buyvalue = $value;
              $cda->tradeflag = $sellok;
              $cda->trade_start = $run_time;
              $cdaorders = $cdaorders + 1;
              $fh = fopen($tradefile, "a") or die("Cannot open file");
              fwrite($fh, "========================== \n");
              fwrite($fh, "Runtime $run_time \n");
              fwrite($fh, "Buy on $key BV:$cda->buyvalue \n");
              fwrite($fh, "========================== \n");
              fclose($fh);
            }
          }
          else
          {
            printf("%-9s",$key);
            print "RSI Check not meeting Minimum, resetting back to Buy Prep\n";
            $cda->tradeflag = $buyprep;
          }
        }

        // Buy Order on CDA placed, do order tracking here to make sure order completes
        if($cda->tradeflag == $sellok)
        {
          // Since we are not placing an order, we just assume it completed
          $cda->tradeflag = $sellready;
        }

        // CDA is sellready and is no longer trending upwards - time to sell
        if($cda->tradeflag == $sellready AND $cda->st->direction!=$up AND $cda->mt->direction!=$up)
        {
          // Assume we sell at the current value and not sell if not meeting a minimum
          $cdabuff = $cda->buyvalue * $sellbuffer;
          if($value > $cdabuff)
          {
            $cda->sellvalue = $value;
            $cda->tradeflag = $selldone;
          }
          else
          {
            printf("%-9s",$key);
            print "Did not meet minimum sell amount\n";
          }
        }
		
		// Stop Loss in the new -> notation
		if($stoplossbuffer > 0)     // if == 0 it means SL is disabled
		{
			$cdastoploss = $cda->buyvalue * $stoplossbuffer;
			if($cda->tradeflag == $sellready AND ($value < $cdastoploss))
			{
				printf("%-9s",$key);
				print "hit the stop loss buffer, so we are selling out\n";
				$cda->sellvalue = $value;
				$cda->tradeflag = $selldone;		
			}	
		}		

        // CDA is selldone
        if($cda->tradeflag == $selldone)
        {
          // Sell Order on CDA placed, do order tracking here to make sure order completes
          // Since we are not placing an order, we just assume it completed
          $q = round(((($cda->sellvalue - $cda->buyvalue)/$cda->buyvalue)*100),3);
          $tpc = $q - 0.2;
          $rpc = round($rpc + ($tpc/$maxorders),3);
          $cda->lasttrade = 1;
          $cda->lasttpc = $tpc;
          $cda->tradeflag = $buyready;
          array_push($cda->trade_cycles, ($run_time - $cda->trade_start)); 
          $cda->trade_cycle_avg = round((array_sum($cda->trade_cycles)/count($cda->trade_cycles)),3);
          $cdaorders = $cdaorders - 1;
          printf("%-9s",$key);
          print "Sell Done BV:$cda->buyvalue SV:$cda->sellvalue TPC:$tpc ATC:$cda->trade_cycle_avg\n";
          $fh = fopen($tradefile, "a") or die("Cannot open file");
          fwrite($fh, "========================== \n");
          fwrite($fh, "Runtime $run_time \n");
          fwrite($fh, "Sell Done on $key BV:$cda->buyvalue SV:$cda->sellvalue TPC:$tpc RPC:$rpc ATC:$cda->trade_cycle_avg\n");
          fwrite($fh, "========================== \n");
          fclose($fh);
        }

      }
    }
  }
  if ($replay == 0) sleep($interval);
}
?>

#420

VM Ware will do what you’re asking. I have to admit I ditched VM Ware a few years back in favor of Virtual Box by Oracle. It’s open source and free and I don’t have to muck with maintaining license keys.

https://www.virtualbox.org/

I use Virtual Box on macOS to run Linux and Windows (and a few other OS’s).

Then there’s also Docker, which allows one to compose reusable/redeployable containers. For example, here’s the MongoDB container: https://hub.docker.com/r/bitnami/mongodb/

Docker containers do take some skill and getting used to, so if you’re completely new to virtual machines or command line shells, then its best to start with either VM Ware or Virtual Box.


#421

Great Trade Strategy! Thank you.


#422

Thanks for the macOS version @mwlang. I downloaded Virtual Box 5.2, installed it, and deployed Linux Ubuntu 64bit with dedicated 5GB of memory to it, 50GB VDI Drive, as I am dedicating this laptop to this bot only.


#423

…is this still an option? I am following evey step so I can fish for myself, but at certain moments I feel like I’m a farmer being asked to explain derivatives :sweat_smile: