Experiments

 There are six agents and two types of goods in our experiments. Each agent starts with random endowment of both goods. Prices are in units of good 2; thus, all auction activities are for good 1.

  

Figure 4: Agent 1's performance over time

Each agent has the same CES utility function as defined in (4). We let $\alpha_j =1$ for all j, and $\rho = {1 \over 2}$, and so the utility function becomes $U(x)=\biggl(\sum _j (x_j)^{1 \over 2}\biggr)^2$. According to (5) and (6) we have

Thus the reservation prices are

We test four types of agents: the competitive agent, 0-level agent, 1-level agent and 2-level agent. We put these agents in four kinds of environments where all other agents are: (1) competitive agents; (2) 0-level learning agents; (3) 1-level learning agents; (4) 2-level learning agents. Just to reiterate our definitions, a competitive (o-level naive) agent does not use any information about other agents. It choose its action based on its individual optimization problem. A 0-level learning agent makes no assumption about other agents but use the data of other agents' actions. A 1-level agent assumes other agents are 0-level naive (competitive) agents. A 2-level agent assumes other agents are 1-level agents.

In each of the four environments, we randomly configure the initial endowment of all agents. For each configuration, we change the first agent for four different types, run the experiments for each type and compare the performance of the first agent for each type it assumes. Such procedures are repeated for 50 runs in order to get an average result.

Figure 6 shows the experimental results in one configuration. Note that one cannot draw any conclusions from the magnitude of utility differences, since utility measures are only ordinally scaled.

  

Figure 5: Performance of Agent 1 when assuming different types

  

Figure 6: Performance of Agent 1 when assuming different types

  

Figure 7: Performance of Agent 1 when assuming different types

  

Figure 8: Performance of Agent 1 when assuming different types

Figure 5 presents results for an environment where all other agents are competitive agents. For this environment, we predict that a 1-level agent will perform the best because the it has correct assumption about other agents of their being competitive. As we can see, out of 88% of all experiments, the 1-level agent ranks among the top two types. The 0-level agent has similar performance only 70% of time. The 2-level agent competitive agents perform much worse than the other types.

Figure 6 presents results for an environment where all other agents are 0-level agents. For this environment, we would not expect 1-level or 2-level agents to perform better than the 0-level agent because these agents make incorrect assumptions about their counterparts. From Figure 2, we see that the 0-level agent, which achieves first- or second-best performance 82% of the time, is clearly the best among the types. The competitive agent still performs the worst.

When we change environments to one in which all other agents are 1-level, the experimental results (Figure 7) show that the 2-level agent performs worse than the 1-level agent, but almost as well as the 0-level agent. This is a little unexpected. We expected that the 2-level agent should do well since it correctly assumes other agents are 1-level agents. Our explanation is that the 2-level agent we implemented is a simplified version of a full 2-level agent, whose model should include all possible variables. Computationally, this is not feasible, and it remains an open questions how to form a computationally efficient and correct model of 2-level agent.

Figure 8 shows the results of an environment where all other agents are 2-level agents. In this environment, we did not expect either 1-level and 2-level agent to perform well, because these agents make incorrect assumptions about the other agents. The experimental results confirm our expectation. Both 1-level and 2-level agents perform worse than the 0-level agent, who achieves first- or second-best performance 73

Our experiments suggest that, in most cases, the 0-level learning agent performs better than any the other types, especially when these others make incorrect assumptions about the environment. Our experiments also show that an agent with correct assumptions can do better than the agent with no assumptions, for example, the 1-level agent performs best when other agents are competitive. Finally, our experiments show that the competitive agent performs worst in all four cases. This suggests that if an agent does not use any information about other agents in the system, the agent will perform worse than those who do use the information.

Our results differ from Santa Fe Tournament [19] where agents with an simple, non-adaptive strategy perform best. That simple strategy is as following: wait in the background and let the others do the negotiation, but when bid and ask get sufficiently close, jump in and steal the deal. This reveals the difference between that simple agent and our competitive agents. First, there is no negotiation process in our auction setup, so there is no way an agent can jump in at last moment. Second, our competitive agent is a true-telling agent who never takes advantage of other agents' bids. The Santa Fe winning agent actually forms a strategy against what other agents do, thus it should be called a strategic agent instead of ``simple'' agent. The fact that this agent monitors other agents' aggregate behavior, the final bid and ask, makes it fall into the 0-level agent category in our paper.

In a few runs of our experiments, all the types of learning agents perform worse than the competitive agent. This confirms our result on conjectural equilibrium [29] where learning can lead to suboptimal outcome.