Problem Difficulty

The program out-performs human experts and significantly out-performs non-experts:

• Expert who knows subroutine library: 30 minutes

• Non-expert: several days

• Program: 3 minutes

Time to construct specification:

• Expert: a few minutes

• Non-expert: 30 minutes