2007 TAC-SCM Prediction Challenge


Jump to the results of the final round.


Jump to the results of the qualifying round.

Welcome to the home of the 2007 TAC-SCM Prediction Challenge. In this challenge, agents will make daily predictions over the course of a number of TAC SCM games about four different types of prices: current and future computer prices, and current and future component prices. Instead of making predictions about live TAC SCM games in which they are participating, agents will make predictions for games that are "re-played" from game logs on behalf of another agent named SCMPredictionAgent.

Schedule:
Monday, July 2: Qualifying round
Tuesday, July 24: Final round

The full specifications are available here. (Updated June 18.)

The following is the beta release of the challenge software. Feel free to email us with any questions or comments.

Prediction Challenge Software
Download the software for parsing game logs and processing predictions, as well as sample prediction code:
standalone version (useful for agent development)
client/server version (to be used in the competition)
View the JavaDocs.

SCMPredictionAgent
Download the source and binary of the SCMPredictionAgent (the TAC SCM agent for which predictions will be made) here.
This agent is based on the TacTex Starter Agent.

Sample game logs
Download 5 sample game logs here (28MB).
Download 16 more sample game logs here (92MB). (Agents: SCMPredictionAgent, PhantAgent05, PhantAgent06, TacTex06, Maxon06, Mertacor05)


Qualifying Round
The qualifying round consisted of 3 groups of 16 games. In addition to the SCMPredictionAgent, each group contained the following agents (from the TAC SCM repository):

A: DeepMaize06, RationalSCM05, Foreseer05, PhantAgent05 x 2
B: MinneTAC06, TacTex05, TacTex06, Maxon06 x 2
C: GoBlueOval05, Foreseer05, GeminiJK05, Mertacor05, MinneTAC05

Download the game logs (253MB).

Results: 

Average rank:
1. DeepMaize:  1.75
2. TacTex:     2
3. Botticelli: 2.5
4. Kshitij:    3.75

Current computer prices RMS error:
1. DeepMaize:  0.04632
2. TacTex:     0.04812
3. Kshitij:    0.04882
4. Botticelli: 0.09449

Future computer prices RMS error: 
1. TacTex:     0.09318
2. DeepMaize:  0.09473
3. Botticelli: 0.13984
4. Kshitij:    0.15428

Current component prices RMS error: 
1. Botticelli: 0.04163
2. TacTex:     0.04239
3. DeepMaize:  0.04810
4. Kshitij:    0.10654

Future component prices RMS error: 
1. DeepMaize:  0.09925
2. Botticelli: 0.10067
3. TacTex:     0.10714
4. Kshitij:    0.16324


Final Round
The final round consisted of 3 groups of 16 games. In addition to the SCMPredictionAgent, each group contained the following agents (from the TAC SCM repository):

A: PhantAgent06, Maxon06, RationalSCM05, Tiancalli06, PhantAgent05
B: GoBlueOval05, GeminiJK05, RationalSCM05, PhantAgent05, TacTex06
C: Maxon06, MinneTAC05, DeepMaize05, Foreseer05, PhantAgent06

Download the game logs (247MB).

Results: 

Average Rank:
1. DeepMaize:  1.5
2. TacTex:     2
3. Botticelli: 2.5
4. Kshitij:    4

Current computer prices RMS error:
1. TacTex:     0.04554
2. DeepMaize:  0.04682
3. Botticelli: 0.04714
4. Kshitij:    0.04873

Future computer prices RMS error: 
1. TacTex:     0.09156
2. DeepMaize:  0.09586
3. Botticelli: 0.10238
4. Kshitij:    0.11094

Current component prices RMS error: 
1. DeepMaize:  0.03919
2. Botticelli: 0.04168
3. TacTex:     0.04284
4. Kshitij:    0.13333

Future component prices RMS error: 
1. DeepMaize:  0.09427
2. Botticelli: 0.09700
3. TacTex:     0.10338
4. Kshitij:    0.13886