This part of the tutorial will cover more advanced topics that build upon our previous knowledge of constructing Python algorithms for trading.
First, let's go over the 'set)universe function.
def initialize(context)
set_universe(universe.DollarVolumeUniverse(99.0, 100.0))
def handle_data(context, data):
symbol('GOOG')
for stock in data:
print len(data)
So from this point on, any stocks or securities that you inquiry about using functions will be automatically included into our stocks universe.
Now, let's go over the 'fetch_csv()' function. This function allows us to import an external dataset in the CSV format.
fetch_csv("location of your csv file"
pre_func=preview,
date_column='data,
universe_func=(my_universe))
context.target_weight = 0.01
def handle_data(context, data)
the fetch_csv("location of your csv file" will import your csv file full of securities, ticker symbols, dates, etc, into our algorithm. date_column='data' will locate and import the date column of your securities. And finally it will use the my_universe function to define our universe of securities.
What our algorithm is going to do is take our csv file and convert all of our securities into SID objects sids = set(financials['sid']) and then pass it through to our my_universe function defined as def my_universe(context, fetcher_data): and then that is going to grab all financial securities in our financials fetcher data financials = fetcher_data[fetcher_data['GICS Sector'] === 'Financials'], so it will grab all of the unique symbols of those securities, and then finally we are going to get the universe size so that we know how much weight to put into each security context.target_weight = 1.0/context.count and then we will return the set we got previously sids = set(financials['sid'] and return it via return sids.
So, the function financials = fetcher_data[fetcher_data['GICS Sector'] === 'Financials'] is describing all the securities listed in the S&P 500, which we will then grab their unique SIDs via the sids = set(financials['sid'] function, and then it will return that list return sids, and then we will use that as our daily universe for the stock. The pre_func=preview is a function that will allow us to preview the outcome of algorithm. The df['data'] = '11/1/15' function will allow us to pull those dates directly from our csv file or we can view them manually in Excel. The point of using the function is to tell our algorithm how to associate the SID object to the date that it belongs to.
We are then ordering that security with the context.target_weight which we found by calculating the number of securities in our universe and dividing that by 1 context.target_weight = 1.0/context.count .
**NOTE: this is all in daily mode, if you want it in minutely mode use the schedule_function(func=rebalance, day_rule=day_rules.every_day(), time_rule=time_rules.market_open()) def rebalance(context, data): **
**NOTE: the maximum number of securities that you can import from a csv file into your universe is going to be around 200! 82 is the typical number of stocks in a 1% range of the dollar volume universe. You can also define your universe with a mix of fundamental factors.
Here is what our code will look like:
import datetime
import pandas as pd
import numpy as np
def preview(df):
log.info(' \n %s ' % df.head())
df['data'] = '11/1/15'
return df
# Function for returning a set of SIDs from fetcher_data
def my_universe(context, fetcher_data):
# Grab just the SIDs for the Financials sector w/in the SP500:
financials = fetcher_data[fetcher_data['GICS Sector'] == 'Financials']
sids = set(financials['sid'])
symbols = [s.symbol for s in sids if s != 0]
context.count = len(symbols)
print "total universe size: {c}".format(c=context.count)
# Compute target equal-weight for each stock in the SP500 Financials
universe
context.target_weight = 1.0/context.count
return sids
def initialize(context):
# Fetch the SP500 constituents -- sourced from Quandl
# https://s3.amazonaws.com/quandl-static-content/Ticker+CSV
%27s/Indicies/SP500.csv
# static snapshot of the SP500 constituents as of 10/2013, along with
GICS sectors
# I'm grabbing the data from a file on my dropbox folder which I
modified by adding
# a date column, alternatively you could add the date inside of a
more complicated
# pre_func and use the csv file as is.
fetch_csv(
"https://dl.dropboxusercontent.com/u/169032081/SP500.csv",
pre_func=preview,
date_column='date',
universe_func=(my_universe))
context.target_weight = 0.01
schedule_function(func=rebalance,
date_rule=date_rules.every_day(),
time_rule=time_rules.market_open())
def rebalance(context, data):
# Loop over every stock in the Financials sector and make sure we have an equal-weight
# exposure using the order_target_percent() method.
for stock in data:
# Guard for missing stock data
if 'price' in data[stock]:
order_target_percent(stock,context.target_weight)
else:
log.warn("No price for {s}".format(s=stock))
def handle_data(context,data):
pass
Sunday, January 31, 2016
Saturday, January 30, 2016
Quantopian Introduction, part 4
Welcome to part 4 of my Quantopian tutorial! In this tutorial we will go over how to use fundamentals data!
This tutorial is going to cover how to select a dynamic portfolio from fundamentals data. This means that we will be selecting securities based upon a property like market cap rather than just hardcoding a list at the beginning of the algorithm. Dynamic portfolio selection leads to more robust algorithms as it reduces the chances of over-fitting your strategy to a particular backtest period. We can use the get_fundamentals function to access the skeleton of a fundamentals query featured below.
import numpy as np
# This function is run once at the beginning of the algorithm, REQUIRED
def initialize(context):
# AAPL stock
# context.stock = sid(24)
# SPY stock
# context.market = sid(8554)
schedule_function(check_mean, date_rules.every_day(), time_rules.market_close(minutes=30))
# This function is run once per day before any calls to handle_data, OPTIONAL
def before_trading_start(context, data):
get_fundamentals(
#Fundamentals Reference Page: https://www.quantopian.com/help/fundamentals
context.fundamental_df = get_fundamentals(
query(
fundamentals.income_statement.total_revenue
)
.filter(
fundamentals.valuation.market_cap > 30000000000
)
.order_by(
fundamentals.valuation.market_cap.desc()
)
.limit(10)
)
update_universe(context.fundamental_df.columns.values)
Ok, so let's explain what all this code is above us. First we have context.fundamental_df = fundamentals(query(, what this does is open up the fundamentals skeleton that will allow us to choose our criteria and values per category. An example of this would be the .fundamentals.valuation.market_cap > 3000000000 which tells the algorithm "hey look, I only want to include companies with a market cap of greater than 300 billion dollars!"
The query( section at the beginning of our algorithm on the second line above, tells the algorithm what data we actually want to access. As you can see, we are telling our algorithm to look at the income statement of the company to determine whether or not it is a 300 billion dollar company!
The .order_by( section will allow us to sort the results of our query, and in this situation we are going to sort them by market-cap in descending order.
And finally, our .limit(10) will limit the number of securities returned, in this case we are limiting the securities to 10.
Also, we are going to store our results (information) in a dataframe denoted by the df listed above and we are giving it the name context.fundamental. The results will be a pandas dataframe where the columns are securities and the rows are the fundamental values that we queried for.
update_universe(context.fundamental_df.columns.values) , this will update our securities universe by adding in our information that we queried above, notice how the context.fundamental_df is the name we gave to our query for market cap info? It is indeed the same!
# This function is run once per bar, REQUIRED
def handle_data(context, data):
pass
def check_mean(context, data):
print get_datetime('US/Eastern')
# Get the close price of each of the last 10 minutes for all stocks
# in our universe.
hist = history(10, '1m', 'close_price')
# Print the 10-minute mean close price for each stock in our universe.
for stock in data:
print '%s - mean price: %.2f, total revenue: %.2f' % (stock.symbol, np.mean(hist[stock]), context.fundamental_df[stock][0])
print '-------------------------------'
print ''
The print '%s - mean price: %.2f, total revenue: %.2f' % (stock.symbol, np.mean(hist[stock]) function will go through and check the mean of the information we have gathered, and then it will print it for us so we can look through it. Additionally, it will print our total revenue through the use of the context.fundamental_df[stock][0] function.
And that's it! You have now mastered the ability to build your own algorithm! Next lesson, we will move into more complex and powerful functions that will allow us to check a greater amount of information using mathematical tools!
This tutorial is going to cover how to select a dynamic portfolio from fundamentals data. This means that we will be selecting securities based upon a property like market cap rather than just hardcoding a list at the beginning of the algorithm. Dynamic portfolio selection leads to more robust algorithms as it reduces the chances of over-fitting your strategy to a particular backtest period. We can use the get_fundamentals function to access the skeleton of a fundamentals query featured below.
import numpy as np
# This function is run once at the beginning of the algorithm, REQUIRED
def initialize(context):
# AAPL stock
# context.stock = sid(24)
# SPY stock
# context.market = sid(8554)
schedule_function(check_mean, date_rules.every_day(), time_rules.market_close(minutes=30))
# This function is run once per day before any calls to handle_data, OPTIONAL
def before_trading_start(context, data):
get_fundamentals(
#Fundamentals Reference Page: https://www.quantopian.com/help/fundamentals
context.fundamental_df = get_fundamentals(
query(
fundamentals.income_statement.total_revenue
)
.filter(
fundamentals.valuation.market_cap > 30000000000
)
.order_by(
fundamentals.valuation.market_cap.desc()
)
.limit(10)
)
update_universe(context.fundamental_df.columns.values)
Ok, so let's explain what all this code is above us. First we have context.fundamental_df = fundamentals(query(, what this does is open up the fundamentals skeleton that will allow us to choose our criteria and values per category. An example of this would be the .fundamentals.valuation.market_cap > 3000000000 which tells the algorithm "hey look, I only want to include companies with a market cap of greater than 300 billion dollars!"
The query( section at the beginning of our algorithm on the second line above, tells the algorithm what data we actually want to access. As you can see, we are telling our algorithm to look at the income statement of the company to determine whether or not it is a 300 billion dollar company!
The .order_by( section will allow us to sort the results of our query, and in this situation we are going to sort them by market-cap in descending order.
And finally, our .limit(10) will limit the number of securities returned, in this case we are limiting the securities to 10.
Also, we are going to store our results (information) in a dataframe denoted by the df listed above and we are giving it the name context.fundamental. The results will be a pandas dataframe where the columns are securities and the rows are the fundamental values that we queried for.
update_universe(context.fundamental_df.columns.values) , this will update our securities universe by adding in our information that we queried above, notice how the context.fundamental_df is the name we gave to our query for market cap info? It is indeed the same!
# This function is run once per bar, REQUIRED
def handle_data(context, data):
pass
def check_mean(context, data):
print get_datetime('US/Eastern')
# Get the close price of each of the last 10 minutes for all stocks
# in our universe.
hist = history(10, '1m', 'close_price')
# Print the 10-minute mean close price for each stock in our universe.
for stock in data:
print '%s - mean price: %.2f, total revenue: %.2f' % (stock.symbol, np.mean(hist[stock]), context.fundamental_df[stock][0])
print '-------------------------------'
print ''
The print '%s - mean price: %.2f, total revenue: %.2f' % (stock.symbol, np.mean(hist[stock]) function will go through and check the mean of the information we have gathered, and then it will print it for us so we can look through it. Additionally, it will print our total revenue through the use of the context.fundamental_df[stock][0] function.
And that's it! You have now mastered the ability to build your own algorithm! Next lesson, we will move into more complex and powerful functions that will allow us to check a greater amount of information using mathematical tools!
Friday, January 29, 2016
Quantopian Introduction, part 3
This is the third part of our Quantopian tutorial! Let's dive in!
#this function is run once per bar, REQUIRED
def handle_data(context, data);
Data variable and trading universe will be the topic we will start with.
A trading universe is any stock that we might be interested in trading in a given bar in our algorithm.
The trading universe is represented by the data variable, which is a dictionary key'd by securities.
def handle_data(context, data);
log.info(get_datetime())
for stock in data;
for stock in data;
log.info('Close price for %s: .2f' % (stock.symbol, data[stock].close_price))
log.info('/n')
So what we did here is logged all this so we can take a look at the minute/lead/close price of each our securities ('Close price for (stock.symbol, data[stock].close_price)), and then we added in some formatting so that it all prints out in a more readable form (%s: .2f' %). And then we added the print time so we can see which minute we are on ( log.info(get_datetime()) ) And then we printed a new line for spacing ( log.info('/n') ).
Hit the 'Build Algorithm' key and what you will find is the pricing data for AAPL and SPY in each minute.
The 'stock universe' includes any stocks that we currently hold position in, as well as any stocks that we have explicitly referenced in the context variable. Another way to get a stock in our universe is to use the 'update universe function'.
Now let's turn our attention to the 'history()' function! The history function is used to us historical data for each stock in our stock universe.
#this function is run once per bar, REQUIRED
def handle_data(context, data);
def handle_data(context, data);
print history(5, '1m', 'close_price')
What this is doing is grabbing the history for our SPY and APPL data for the last 5 minute bars, and then we put '1m' for our frequency, and then we put close_price as the field that we are interested in. This returns as a pandas data frame with each column being a different stock and each row being a different date-time.
If we wanted a days frequency we would simply put '1d' instead of '1m'. Additionally, we can also change our field, so that we can find the volume traded rather than the closing price. We do this by placing 'volume' where our 'closing_price' is.
If we want to find the ten day mean in the closing price of our stocks, we can change the number of bars to '11' and keep the frequency at '1d' and then I am actually going to save this function which, we can do through the following function: hist = history(11, '1d', 'close_price')
Now we want to make a 'for' loop going through each stock in data:
#this function is run once per bar, REQUIRED
def handle_data(context, data);
def handle_data(context, data);
hist = history(11, '1d', 'close_price')
for stock in data:
print '10-day mean close price for stock %s: %2f' % (stock.symbol, np.mean(hist[stock][:-1]))
So, what we did here was print a line of text '10-day mean close price for stock' that references our stock %s: %2f' % and then we retrieve the stock symbol, and then we import the numpy library denoted as np (**NOTE: when importing a library you must also type a line of a code at the very beginning of your algorithm (above the def initialize(context) function); so above that function we would have the following line of code: import numpy as np. )
followed by the use of the statistical .mean variable of the (hist <--- our historical data for our stock. And then we type [stock] for the column that corresponds to the specific stock in our loop.
The [:-1] portion of our code excludes the last row of our column.
And the result will give us our ten day mean for AAPL and SPY.
We can change the our days to minutes like this: '10-minute mean...'
Thank you for reading part 3 of our tutorial, and as always if you have any questions or comments please send me an e-mail or comment!
The [:-1] portion of our code excludes the last row of our column.
And the result will give us our ten day mean for AAPL and SPY.
We can change the our days to minutes like this: '10-minute mean...'
Thank you for reading part 3 of our tutorial, and as always if you have any questions or comments please send me an e-mail or comment!
Thursday, January 28, 2016
Quantopian Introduction, part 2
This is part 2 in our tutorial demonstrating how to use the online interactive investment trading tool known as Quantopian, using Python. These are the most basic code inputs that are either REQUIRED or they are OPTIONAL; the optional ones are encouraged if you have information to query before running the actual trading algorithm.
#this function is run once at the beginning of the algorithm REQUIRED
def initialize(context)
context.stock = sid(24)
pass
#this function is run once per day before any calls to handle_data OPTIONAL
def before_trading_start(context, data)
pass
^**This is typically where we will run calculations necessary for pre-computations that will be used for the rest of the day.
#this function is run once per bar, REQUIRED
def handle_data(context, data);
pass
^**Ok, so this bit of code right here is retrieves data either by the minute or by the day, depending on the variable you set. Speaking of variables, the command 'handle_data' requires two inputs: context, and data.
Now let's go over securities referencing, SID, or "Security Identifier". The SID function takes care of the issues that might arise when backtesting an algorithm over an extended period of time, including if the stock ticker symbol changes. That's right, the SID function will keep the correct stock that you're using.
An example would be our previous function that we used above:
context.stock = sid(24)
..... sid(24) means that the function is calling upon the Apple stock symbol (AAPL). the (24) is the ID number given to the AAPL stock, so rather than having AAPL as the stock symbol, Quantopian identifies it by the (24) integers.
The context.stock, is a function used to store the stock data.
Next, we can 'print' out the function using the following:
#this function is run once per bar, REQUIRED
def handle_data(context, data);
print context.stock
Notice how this is mostly the same list of functions we used previously? The only difference is that we removed the 'pass' function and replaced it with the 'print context.stock'
A really cool feature of Quantopian with Python in this instance is that we don't have to use the ID number (SID) if we don't want to. We can simply use the following bit of code to locate the stock by it's ticker symbol:
symbol('AAPL')
**NOTE: the ticker symbol is not robust to ticker changes; in other words, if the ticker symbol changes the function will not change with it, rendering it useless. This is not a dynamic function so it is best to use the SID function instead!
Now we are going to go over how to order securities!
The portfolio value in some contexts, including this one, is calculated as the sum value of all open positions plus the cash balance.
Let's say we want half our portfolio value to be long Apple stock
#this function is run once per bar, REQUIRED
def handle_data(context, data);
order_target_percent(context.stock, 0.5)
This takes two parameters, the first being the stock that we want to order and the second being the target percent. The 0.5 dictates that it will take half of our available funds (0.5) and order Apple stock. So if we have $100,000 available and no open positions, this function will go through and order $50,000 of Apple stock.
Now, let's say we want to hedge our investment, we can go short in SPY, a stock that mirrors the performance of the S&P 500 index, for the other half of our portfolio.
#this function is run once at the beginning of the algorithm REQUIRED
def initialize(context)
context.stock = sid(24)
context.market = sid(8554)
The (8554) is a reference to the SPY security.
Notice that we added the 'context.market = sid(8554)' to our previous function.
#this function is run once per bar, REQUIRED
def handle_data(context, data);
order_target_percent(context.stock, 0.5)
order_target_percent(context.market, -0.5)
Notice how we added 'order_target_percent(context.market, -0.5)' to our previous function? This algorithm is saying 'I want to short the SPY using half of my portfolio (-0.5). The short position is denoted by the negative ("-") symbol before the (0.5) number.
I hope you enjoyed this brief tutorial! Part 3 will be added shortly!
Let me know if you have any questions or comments!
#this function is run once at the beginning of the algorithm REQUIRED
def initialize(context)
context.stock = sid(24)
pass
#this function is run once per day before any calls to handle_data OPTIONAL
def before_trading_start(context, data)
pass
^**This is typically where we will run calculations necessary for pre-computations that will be used for the rest of the day.
#this function is run once per bar, REQUIRED
def handle_data(context, data);
pass
^**Ok, so this bit of code right here is retrieves data either by the minute or by the day, depending on the variable you set. Speaking of variables, the command 'handle_data' requires two inputs: context, and data.
Now let's go over securities referencing, SID, or "Security Identifier". The SID function takes care of the issues that might arise when backtesting an algorithm over an extended period of time, including if the stock ticker symbol changes. That's right, the SID function will keep the correct stock that you're using.
An example would be our previous function that we used above:
context.stock = sid(24)
..... sid(24) means that the function is calling upon the Apple stock symbol (AAPL). the (24) is the ID number given to the AAPL stock, so rather than having AAPL as the stock symbol, Quantopian identifies it by the (24) integers.
The context.stock, is a function used to store the stock data.
Next, we can 'print' out the function using the following:
#this function is run once per bar, REQUIRED
def handle_data(context, data);
print context.stock
Notice how this is mostly the same list of functions we used previously? The only difference is that we removed the 'pass' function and replaced it with the 'print context.stock'
A really cool feature of Quantopian with Python in this instance is that we don't have to use the ID number (SID) if we don't want to. We can simply use the following bit of code to locate the stock by it's ticker symbol:
symbol('AAPL')
**NOTE: the ticker symbol is not robust to ticker changes; in other words, if the ticker symbol changes the function will not change with it, rendering it useless. This is not a dynamic function so it is best to use the SID function instead!
Now we are going to go over how to order securities!
The portfolio value in some contexts, including this one, is calculated as the sum value of all open positions plus the cash balance.
Let's say we want half our portfolio value to be long Apple stock
#this function is run once per bar, REQUIRED
def handle_data(context, data);
order_target_percent(context.stock, 0.5)
This takes two parameters, the first being the stock that we want to order and the second being the target percent. The 0.5 dictates that it will take half of our available funds (0.5) and order Apple stock. So if we have $100,000 available and no open positions, this function will go through and order $50,000 of Apple stock.
Now, let's say we want to hedge our investment, we can go short in SPY, a stock that mirrors the performance of the S&P 500 index, for the other half of our portfolio.
#this function is run once at the beginning of the algorithm REQUIRED
def initialize(context)
context.stock = sid(24)
context.market = sid(8554)
The (8554) is a reference to the SPY security.
Notice that we added the 'context.market = sid(8554)' to our previous function.
#this function is run once per bar, REQUIRED
def handle_data(context, data);
order_target_percent(context.stock, 0.5)
order_target_percent(context.market, -0.5)
Notice how we added 'order_target_percent(context.market, -0.5)' to our previous function? This algorithm is saying 'I want to short the SPY using half of my portfolio (-0.5). The short position is denoted by the negative ("-") symbol before the (0.5) number.
I hope you enjoyed this brief tutorial! Part 3 will be added shortly!
Let me know if you have any questions or comments!
Quantopian Introduction
Quantopian.com is pretty neat. You have access to a lot of data, can build your own algorithms, backtest them, and even have the support of the Quantopian community who will share their algorithms with you! Let's get into it here!
This introductory lesson is based upon the Quantopian tutorial, as we advance and move into more sophisticated techniques we will move away from their tutorial however I see fit. My goal is to make this simple and informative so that anyone can learn algorithmic trading without too much of a headache!
Here we go:
_________________________________________________________________________________
This is the default "Sample Mean Reversion" tutorial that I am regurgitating from Quantopian.com and in addition I will provide the reader with a deeper understanding of what's going on and why.
Background: Mean Reversion is a statistical method used in finance: Mean Reversion "is the assumption that a stocks price will tend to move to the average price over time"
"When the current market price is less than the average price, the stock is considered attractive for purchase, with the expectation that the price will rise. When the current market price is above the average price, the market price is expected to fall. In other words, deviations from the average price are expected to revert to the average"-https://en.wikipedia.org/wiki/Mean_reversion_(finance)
In other words, a stock that deviates either above or below it's market price will eventually revert back to it's historical mean. We can visualize this using continuous time-series analysis otherwise known as Orstein-Uhlenbeck process. We don't need to go into depth on the Orstein-Uhlenbeck process, but if you would like to know more see Wikipedia, and basic stochastic processes!
Ok, so now we know what a mean reversion is and why we are doing it. It helps us to understand if a stock is a buy or sell value based upon its current market price compared to its historical mean.
Now for the code: (mind you, this is in Python since we are using Quantopian.com).
**Any line that is preceded with a hashtag (#) and is underlined means that it is a 'comment'-comments are used within lines of code by the creator to define a process, or explain why he/she has done something. You do not need to include it in your code!
# Import the libraries we will use here
import numpy as np
**Ok, so import numpy as np is the coding package we are using in our Python module. This allows us to use certain commands as tools and helps us build the algorithm we intend to build!**
# The initialize function is the place to set your tradable universe and define any parameters.
def initialize(context):
# Use the top 1% of stocks defined by average daily trading volume.
set_universe(universe.DollarVolumeUniverse(99, 100))
# Set execution cost assumptions. For live trading with Interactive Brokers
# we will assume a $1.00 minimum per trade fee, with a per share cost of $0.0075.
set_commission(commission.PerShare(cost=0.0075, min_trade_cost=1.00))
# Set market impact assumptions. We limit the simulation to
# trade up to 2.5% of the traded volume for any one minute,
# and our price impact constant is 0.1.
set_slippage(slippage.VolumeShareSlippage(volume_limit=0.025, price_impact=0.10))
**NOTE: slippage defines how your algorithm will impact the price of the security. As a participant in the market, you will have a small impact on prices depending on how you buy or sell.
# Define the other variables
context.long_leverage = 0.5
context.short_leverage = -0.5
context.lower_percentile = 20
context.upper_percentile = 80
context.returns_lookback = 5
# Rebalance every Monday (or the first trading day if it's a holiday).
# At 11AM ET, which is 1 hour and 30 minutes after market open.
schedule_function(rebalance,
date_rules.week_start(days_offset=0),
time_rules.market_open(hours = 1, minutes = 30))
# The handle_data function is run every bar.
def handle_data(context,data):
# Record and plot the leverage of our portfolio over time.
record(leverage = context.account.leverage)
# We also want to monitor the number of long and short positions
# in our portfolio over time. This loop will check our positition sizes
# and add the count of longs and shorts to our plot.
longs = shorts = 0
for position in context.portfolio.positions.itervalues():
if position.amount > 0:
longs += 1
if position.amount < 0:
shorts += 1
record(long_count=longs, short_count=shorts)
# This rebalancing is called according to our schedule_function settings.
def rebalance(context,data):
# Get the last N days of prices for every stock in our universe.
prices = history(context.returns_lookback, '1d', 'price')
# Calculate the past 5 days' returns for each security.
returns = (prices.iloc[-1] - prices.iloc[0]) / prices.iloc[0]
# Remove stocks with missing prices.
# Remove any stocks we ordered last time that still have open orders.
# Get the cutoff return percentiles for the long and short portfolios.
returns = returns.dropna()
open_orders = get_open_orders()
if open_orders:
eligible_secs = [sec for sec in data if sec not in open_orders]
returns = returns[eligible_secs]
# Lower percentile is the threshhold for the bottom 20%, upper percentile is for the top 20%.
lower, upper = np.percentile(returns, [context.lower_percentile,
context.upper_percentile])
# Select the X% worst performing securities to go long.
long_secs = returns[returns <= lower]
# Select the Y% best performing securities to short.
short_secs = returns[returns >= upper]
# Set the allocations to even weights in each portfolio.
long_weight = context.long_leverage / len(long_secs)
short_weight = context.short_leverage / len(short_secs)
for security in data:
# Buy/rebalance securities in the long leg of our portfolio.
if security in long_secs:
order_target_percent(security, long_weight)
# Sell/rebalance securities in the short leg of our portfolio.
elif security in short_secs:
order_target_percent(security, short_weight)
# Close any positions that fell out of the list of securities to long or short.
else:
order_target(security, 0)
log.info("This week's longs: "+", ".join([long_.symbol for long_ in long_secs.index]))
log.info("This week's shorts: " +", ".join([short_.symbol for short_ in short_secs.index]))
First, in the right hand side, you have your return; that is pretty self-explanatory, it is telling you how much you have made or lost given percentage of movement in the value of the stock.
Next we have alpha; alpha means 'a measure of performance on a risk adjusted basis' and additionally 'The abnormal rate of return on a security or portfolio in excess of what would be predicted by an equilibrium model like the capital asset pricing model (CAPM).' -http://www.investopedia.com/terms/a/alpha.asp
After alpha, we have beta which measures the volatility or risk.
This introductory lesson is based upon the Quantopian tutorial, as we advance and move into more sophisticated techniques we will move away from their tutorial however I see fit. My goal is to make this simple and informative so that anyone can learn algorithmic trading without too much of a headache!
Here we go:
_________________________________________________________________________________
This is the default "Sample Mean Reversion" tutorial that I am regurgitating from Quantopian.com and in addition I will provide the reader with a deeper understanding of what's going on and why.
Background: Mean Reversion is a statistical method used in finance: Mean Reversion "is the assumption that a stocks price will tend to move to the average price over time"
"When the current market price is less than the average price, the stock is considered attractive for purchase, with the expectation that the price will rise. When the current market price is above the average price, the market price is expected to fall. In other words, deviations from the average price are expected to revert to the average"-https://en.wikipedia.org/wiki/Mean_reversion_(finance)
In other words, a stock that deviates either above or below it's market price will eventually revert back to it's historical mean. We can visualize this using continuous time-series analysis otherwise known as Orstein-Uhlenbeck process. We don't need to go into depth on the Orstein-Uhlenbeck process, but if you would like to know more see Wikipedia, and basic stochastic processes!
Ok, so now we know what a mean reversion is and why we are doing it. It helps us to understand if a stock is a buy or sell value based upon its current market price compared to its historical mean.
Now for the code: (mind you, this is in Python since we are using Quantopian.com).
**Any line that is preceded with a hashtag (#) and is underlined means that it is a 'comment'-comments are used within lines of code by the creator to define a process, or explain why he/she has done something. You do not need to include it in your code!
# Import the libraries we will use here
import numpy as np
**Ok, so import numpy as np is the coding package we are using in our Python module. This allows us to use certain commands as tools and helps us build the algorithm we intend to build!**
# The initialize function is the place to set your tradable universe and define any parameters.
def initialize(context):
# Use the top 1% of stocks defined by average daily trading volume.
set_universe(universe.DollarVolumeUniverse(99, 100))
# Set execution cost assumptions. For live trading with Interactive Brokers
# we will assume a $1.00 minimum per trade fee, with a per share cost of $0.0075.
set_commission(commission.PerShare(cost=0.0075, min_trade_cost=1.00))
# Set market impact assumptions. We limit the simulation to
# trade up to 2.5% of the traded volume for any one minute,
# and our price impact constant is 0.1.
set_slippage(slippage.VolumeShareSlippage(volume_limit=0.025, price_impact=0.10))
**NOTE: slippage defines how your algorithm will impact the price of the security. As a participant in the market, you will have a small impact on prices depending on how you buy or sell.
# Define the other variables
context.long_leverage = 0.5
context.short_leverage = -0.5
context.lower_percentile = 20
context.upper_percentile = 80
context.returns_lookback = 5
# Rebalance every Monday (or the first trading day if it's a holiday).
# At 11AM ET, which is 1 hour and 30 minutes after market open.
schedule_function(rebalance,
date_rules.week_start(days_offset=0),
time_rules.market_open(hours = 1, minutes = 30))
# The handle_data function is run every bar.
def handle_data(context,data):
# Record and plot the leverage of our portfolio over time.
record(leverage = context.account.leverage)
# We also want to monitor the number of long and short positions
# in our portfolio over time. This loop will check our positition sizes
# and add the count of longs and shorts to our plot.
longs = shorts = 0
for position in context.portfolio.positions.itervalues():
if position.amount > 0:
longs += 1
if position.amount < 0:
shorts += 1
record(long_count=longs, short_count=shorts)
# This rebalancing is called according to our schedule_function settings.
def rebalance(context,data):
# Get the last N days of prices for every stock in our universe.
prices = history(context.returns_lookback, '1d', 'price')
# Calculate the past 5 days' returns for each security.
returns = (prices.iloc[-1] - prices.iloc[0]) / prices.iloc[0]
# Remove stocks with missing prices.
# Remove any stocks we ordered last time that still have open orders.
# Get the cutoff return percentiles for the long and short portfolios.
returns = returns.dropna()
open_orders = get_open_orders()
if open_orders:
eligible_secs = [sec for sec in data if sec not in open_orders]
returns = returns[eligible_secs]
# Lower percentile is the threshhold for the bottom 20%, upper percentile is for the top 20%.
lower, upper = np.percentile(returns, [context.lower_percentile,
context.upper_percentile])
# Select the X% worst performing securities to go long.
long_secs = returns[returns <= lower]
# Select the Y% best performing securities to short.
short_secs = returns[returns >= upper]
# Set the allocations to even weights in each portfolio.
long_weight = context.long_leverage / len(long_secs)
short_weight = context.short_leverage / len(short_secs)
for security in data:
# Buy/rebalance securities in the long leg of our portfolio.
if security in long_secs:
order_target_percent(security, long_weight)
# Sell/rebalance securities in the short leg of our portfolio.
elif security in short_secs:
order_target_percent(security, short_weight)
# Close any positions that fell out of the list of securities to long or short.
else:
order_target(security, 0)
log.info("This week's longs: "+", ".join([long_.symbol for long_ in long_secs.index]))
log.info("This week's shorts: " +", ".join([short_.symbol for short_ in short_secs.index]))
Ok, so run the code over at Quantopian.com or go ahead and run it through your own local Python IDLE and have a look at the results.
If you're running it over at Quantopian.com and want to know the meaning of your results read here:
Next we have alpha; alpha means 'a measure of performance on a risk adjusted basis' and additionally 'The abnormal rate of return on a security or portfolio in excess of what would be predicted by an equilibrium model like the capital asset pricing model (CAPM).' -http://www.investopedia.com/terms/a/alpha.asp
After alpha, we have beta which measures the volatility or risk.
After beta, we have 'Sharpe', which is the 'Sharpe ratio": it measures the risk-adjusted return. The Sharpe ratio is the average return earned in excess of the risk-free rate per unit of volatility or total risk.
Lastly, we have the drawdown: The peak-to-trough decline during a specific record period of an investment, fund or commodity. A drawdown is usually quoted as the percentage between the peak and the trough.
If you have any questions or concerns please send me an e-mail or post a comment!
Friday, January 8, 2016
Investing for Success: What to do in a down market
One of the scariest thing for a new, amateur investor or even those stock professionals who have been in the business for decades, is a sudden and sharp decline in stocks. Even worse for some folks and organizations who are heavily weighted in stocks, is a long-term decline in overall stock market.
Right now, the stock market is in decline and has been for the past week since it hit it's peak of 17,720.98 on the December 29th, 2015. Since then a lot has happened, Saudi Arabia and Iran are facing their highest tensions in years, the Chinese stock market fell 7% in the first 15 minutes of trading on Monday and continues to decline, and OPEC continues to pump out oil production at the highest rates ever, even in a congested market!
The DJIA right now (as of this writing) is at 16,578. So stocks haven't recovered yet and it could take a month, if not more, before they rebound and normalize again. This is a great lesson in psychology, not so much out of a fearful sort of psychology in people's minds, but of great awareness of their environment (stocks). I am neglected going into detail about the fed rate hike because it frankly is minimal that it has not had an impact on stocks.
First, let's look at why stocks are on the decline here in the U.S.
-As previously mentioned, China. Chinese exports are slowing dramatically as world growth is likewise slowing, having the second largest economy in the world, China makes a huge presence on the world stage, especially when you take into account their manufacturing and heavy industry sectors which are by far the largest in world, producing everything from concrete and copper to IPhones and DVD's.
But.... it's important to note that their stock market (the Shanghai Stock Exchange), is NOT a critical component of their economy, nor is it anything more than an indication of how part of their economy is doing based upon highly speculative trading. This is also NOT how U.S. stock exchanges, specifically the Dow Jones Industrial Average operates. The reason for the differences between the U.S. stock exchange and the Chinese, is that our stock exchange is much more robust and developed, it includes the health and wealth of nearly every large company in the U.S. and every one that has gone public, and with the interconnectivity of our finances and how tied-in our retirement plans such as our 401k's are to our utility of life, everything that happens in the stock market is of some value to someone somewhere, whether or not they think it matters at all.
In China, their stock exchange is not fully developed, not well connected, and their people simply couldn't care less, mostly because it is foreign to them as in it bears no brunt or effect on their daily lives. There are not nearly as many companies listed as in the U.S. and many of them have little foreign investment, even then a lot of them see highly subsidized contracts and help from the Chinese government, and even then their stocks and financial health are not directly linked to Chinese economic performance.
In conclusion to this piece; the Chinese stock market (Shanghai Exchange) is not important, we need not worry about it unless you are directly invested through it. It does not represent the health of the Chinese economy the same way the Dow Jones Industrial Average represents the health of the U.S. economy. So the investors that are freaked out about the Chinese drop in stocks probably already know this, but stocks are about speculation and spatial awareness so they are doing their good-duty of playing it cautionary and pulling out of U.S. companies who have direct links to Chinese manufacturing and investment. It's a safe, but unnecessary strategy.
Were going to look into what we can do as investors here in the U.S. to combat the contagion of fear and also were going to see what we can do to capitalize on the most recent series of events.
------------
GENERAL RULES TO LIVE BY AS AN INVESTOR IN A DOWN MARKET
1. Sit on it, and wait for stocks to rebound and normalize. It might take a week or a year, but they will rebound unless there is something catastrophic, or a series of catastrophes in a specific market you are invested in. Otherwise, just wait it out and use this time as an opportunity to direct your funds typically used for investing in stocks, into debt repayment. Now is a great time to put extra cash down towards repaying your student loans, credit card debt, home equity loans, car payments, etc. The biggest mistake I see from investors is fear and panic, don't be like them it's just a dumb, bad strategy to have.
Let's take a look at some stocks and see what's going on, and what industries have been affected the most.
CHSCP is currently trading at $30.60 with no movement for the day, down from their previous high this week of 30.91. The past week has seen a lot of volatility much like Tesla Motors (TSLA) but to a smaller degree, overall this would be a fun stock to watch. TSLA has seen a steady decline from $233 five days ago to $213 now, with a lot of volatility mixed in. McDonalds (MCD) has been all over the place, their last peak was two days ago with $119, then dropped to $115 and now they're at $116, however if you had held this stock for anymore than a year you would have seen the stock shoot from $93 to it's current $116. That's a fantastic return if you're a long-term investor, and one year is hardly long-term. What about Google? They were $750 five days ago and now down to $717. Their 52 week low is $486, and the 52 week high is $779... ridiculous returns on such a pricey stock!
Pretty much everything is down today. Every sector, industry, everything is down. Bad day for stockholders overall, but even in such a case stocks will rebound and within the market there are some gainers.
2. Short stocks. Shorting is not for everyone and takes a greater understanding of the markets and stock fundamentals then normal stock trading. If you're going to short, you had better be sure that your stock is indeed failing, or have some sort of insurance or hedge against it as most folks will do. In order to short, you must have extra cash laying around that you can put down in case things go bad, because you will need it for the broker depending on the outcome of the short.
3. Continue investing in specific companies through intensive research. This is the riskiest move to make in a down market and only for those investors who are willing to put in the time and research involved in making such a move. You can find gainers in the markets and jump in with the trend and hope the momentum will continue (risky) or go ahead and see if your research leads you to a company that is about to break from the herd and realize some gains.
Take today for example, the best performing stock was that of NGL Energy Partners (NGL) whose stock soared over 45%! But why? If you're thinking it had something to do with the commodities industry or the energy sector you would be wrong. Instead, they sold one of their assets for $350 million, in a move that apparently shook investors the right way. Meanwhile, the rest of the energy sector was mostly meh, not going anywhere and those that did went down. There were a few exceptions like Kinder Morgan (KMI) and others but most every sector overall was down significantly.
------------
Right now, the stock market is in decline and has been for the past week since it hit it's peak of 17,720.98 on the December 29th, 2015. Since then a lot has happened, Saudi Arabia and Iran are facing their highest tensions in years, the Chinese stock market fell 7% in the first 15 minutes of trading on Monday and continues to decline, and OPEC continues to pump out oil production at the highest rates ever, even in a congested market!
The DJIA right now (as of this writing) is at 16,578. So stocks haven't recovered yet and it could take a month, if not more, before they rebound and normalize again. This is a great lesson in psychology, not so much out of a fearful sort of psychology in people's minds, but of great awareness of their environment (stocks). I am neglected going into detail about the fed rate hike because it frankly is minimal that it has not had an impact on stocks.
First, let's look at why stocks are on the decline here in the U.S.
-As previously mentioned, China. Chinese exports are slowing dramatically as world growth is likewise slowing, having the second largest economy in the world, China makes a huge presence on the world stage, especially when you take into account their manufacturing and heavy industry sectors which are by far the largest in world, producing everything from concrete and copper to IPhones and DVD's.
But.... it's important to note that their stock market (the Shanghai Stock Exchange), is NOT a critical component of their economy, nor is it anything more than an indication of how part of their economy is doing based upon highly speculative trading. This is also NOT how U.S. stock exchanges, specifically the Dow Jones Industrial Average operates. The reason for the differences between the U.S. stock exchange and the Chinese, is that our stock exchange is much more robust and developed, it includes the health and wealth of nearly every large company in the U.S. and every one that has gone public, and with the interconnectivity of our finances and how tied-in our retirement plans such as our 401k's are to our utility of life, everything that happens in the stock market is of some value to someone somewhere, whether or not they think it matters at all.
In China, their stock exchange is not fully developed, not well connected, and their people simply couldn't care less, mostly because it is foreign to them as in it bears no brunt or effect on their daily lives. There are not nearly as many companies listed as in the U.S. and many of them have little foreign investment, even then a lot of them see highly subsidized contracts and help from the Chinese government, and even then their stocks and financial health are not directly linked to Chinese economic performance.
In conclusion to this piece; the Chinese stock market (Shanghai Exchange) is not important, we need not worry about it unless you are directly invested through it. It does not represent the health of the Chinese economy the same way the Dow Jones Industrial Average represents the health of the U.S. economy. So the investors that are freaked out about the Chinese drop in stocks probably already know this, but stocks are about speculation and spatial awareness so they are doing their good-duty of playing it cautionary and pulling out of U.S. companies who have direct links to Chinese manufacturing and investment. It's a safe, but unnecessary strategy.
Were going to look into what we can do as investors here in the U.S. to combat the contagion of fear and also were going to see what we can do to capitalize on the most recent series of events.
------------
GENERAL RULES TO LIVE BY AS AN INVESTOR IN A DOWN MARKET
1. Sit on it, and wait for stocks to rebound and normalize. It might take a week or a year, but they will rebound unless there is something catastrophic, or a series of catastrophes in a specific market you are invested in. Otherwise, just wait it out and use this time as an opportunity to direct your funds typically used for investing in stocks, into debt repayment. Now is a great time to put extra cash down towards repaying your student loans, credit card debt, home equity loans, car payments, etc. The biggest mistake I see from investors is fear and panic, don't be like them it's just a dumb, bad strategy to have.
Let's take a look at some stocks and see what's going on, and what industries have been affected the most.
CHSCP is currently trading at $30.60 with no movement for the day, down from their previous high this week of 30.91. The past week has seen a lot of volatility much like Tesla Motors (TSLA) but to a smaller degree, overall this would be a fun stock to watch. TSLA has seen a steady decline from $233 five days ago to $213 now, with a lot of volatility mixed in. McDonalds (MCD) has been all over the place, their last peak was two days ago with $119, then dropped to $115 and now they're at $116, however if you had held this stock for anymore than a year you would have seen the stock shoot from $93 to it's current $116. That's a fantastic return if you're a long-term investor, and one year is hardly long-term. What about Google? They were $750 five days ago and now down to $717. Their 52 week low is $486, and the 52 week high is $779... ridiculous returns on such a pricey stock!
Pretty much everything is down today. Every sector, industry, everything is down. Bad day for stockholders overall, but even in such a case stocks will rebound and within the market there are some gainers.
2. Short stocks. Shorting is not for everyone and takes a greater understanding of the markets and stock fundamentals then normal stock trading. If you're going to short, you had better be sure that your stock is indeed failing, or have some sort of insurance or hedge against it as most folks will do. In order to short, you must have extra cash laying around that you can put down in case things go bad, because you will need it for the broker depending on the outcome of the short.
3. Continue investing in specific companies through intensive research. This is the riskiest move to make in a down market and only for those investors who are willing to put in the time and research involved in making such a move. You can find gainers in the markets and jump in with the trend and hope the momentum will continue (risky) or go ahead and see if your research leads you to a company that is about to break from the herd and realize some gains.
Take today for example, the best performing stock was that of NGL Energy Partners (NGL) whose stock soared over 45%! But why? If you're thinking it had something to do with the commodities industry or the energy sector you would be wrong. Instead, they sold one of their assets for $350 million, in a move that apparently shook investors the right way. Meanwhile, the rest of the energy sector was mostly meh, not going anywhere and those that did went down. There were a few exceptions like Kinder Morgan (KMI) and others but most every sector overall was down significantly.
------------
Subscribe to:
Comments (Atom)