TAC-SCM Prediction Challenge


Jump to the results of the 2008 final round.

Welcome to the home of the 2009 TAC-SCM Prediction Challenge. In this challenge, agents will make daily predictions over the course of a number of TAC SCM games about four different types of prices: current and future computer prices, and current and future component prices. Instead of making predictions about live TAC SCM games in which they are participating, agents will make predictions for games that are "re-played" from game logs on behalf of another agent named SCMPredictionAgent.

An analysis of the 2007 results can be found here:
David Pardoe and Peter Stone. The 2007 TAC SCM Prediction Challenge. In AAAI 2008 Workshop on Trading Agent Design and Analysis, July 2008. PDF  PS 


Schedule:
Qualifying round: June 15-19, 2009 (Monday-Friday) (You only need to run your agent once during the week.)
Final round: Tuesday, July 14 2009 15:00 GMT - 23:00 GMT (8:00 AM - 4:00 PM Pasadena time)

The full specifications are available here.

The following is the Prediction Challenge software, unchanged from 2007. Feel free to email us with any questions or comments.

Prediction Challenge Software
Download the software for parsing game logs and processing predictions, as well as sample prediction code:
standalone version (useful for agent development)
client/server version (to be used in the competition)
View the JavaDocs.

SCMPredictionAgent
Download the source and binary of the SCMPredictionAgent (the TAC SCM agent for which predictions will be made) here.
This agent is based on the TacTex Starter Agent.

Sample game logs
Download 5 sample game logs here (28MB).
Download 16 more sample game logs here (92MB). (Agents: SCMPredictionAgent, PhantAgent05, PhantAgent06, TacTex06, Maxon06, Mertacor05)



Results of the 2008 Final Round
The final round consisted of 3 groups of 16 games. In addition to the SCMPredictionAgent, each group contained the following agents (from the TAC SCM repository):

A: PhantAgent05 GoBlueOval05 Mertacor05 Southampton05 Southampton05
B: GeminiJK05 GeminiJK05 TacTex05 Mertacor05 PhantAgent07
C: RationalSCM05 PhantAgent06 DeepMaize07 Foreseer05 Botticelli05

Download the game logs (253MB).

Results:

Average Rank:
1. TacTex:     1.5
2. DeepMaize:  2
3. Botticelli: 2.5

Current computer prices RMS error:
TacTex:     0.04779
DeepMaize:  0.05130
Botticelli: 0.05310

Future computer prices RMS error:
TacTex:     0.08726
DeepMaize:  0.09750
Botticelli: 0.09934

Current component prices RMS error:
DeepMaize:  0.03476
TacTex:     0.03513
Botticelli: 0.04029

Future component prices RMS error:
Botticelli: 0.09964
TacTex:     0.10225
DeepMaize:  0.10281