Hugging Face's logo

moonshotai
/
Kimi-K2.5

Image-Text-to-Text
Transformers
Safetensors
kimi_k25
feature-extraction
compressed-tensors
conversational
custom_code
Eval Results
Model card Files
xet
 Community
127
Elron commited on
Commit
06c30df
·
verified ·
1 Parent(s): 4d01dfe

Add Open Agent Leaderboard evaluation results

Browse files

Results from the Open Agent Leaderboard (https://www.exgentic.ai), evaluating this model across 6 benchmarks with 5 different agent architectures. See https://huggingface.co/datasets/open-agent-leaderboard/results for the full benchmark.

Files changed (5) hide show
  1. .eval_results/open_agent_leaderboard_claude_code.yaml +56 -0
  2. .eval_results/open_agent_leaderboard_openai_solo.yaml +56 -0
  3. .eval_results/open_agent_leaderboard_smolagents_code.yaml +56 -0
  4. .eval_results/open_agent_leaderboard_tool_calling.yaml +56 -0
  5. .eval_results/open_agent_leaderboard_tool_calling_with_shortlisting.yaml +56 -0
.eval_results/open_agent_leaderboard_claude_code.yaml ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: open-agent-leaderboard/results
3
+ task_id: overall
4
+ value: 0.3026
5
+ source:
6
+ url: https://www.exgentic.ai
7
+ name: Open Agent Leaderboard
8
+ notes: 'agent: Claude Code'
9
+ - dataset:
10
+ id: open-agent-leaderboard/results
11
+ task_id: appworld
12
+ value: 0.08
13
+ source:
14
+ url: https://www.exgentic.ai
15
+ name: Open Agent Leaderboard
16
+ notes: 'agent: Claude Code'
17
+ - dataset:
18
+ id: open-agent-leaderboard/results
19
+ task_id: browsecomp_plus
20
+ value: 0.56
21
+ source:
22
+ url: https://www.exgentic.ai
23
+ name: Open Agent Leaderboard
24
+ notes: 'agent: Claude Code'
25
+ - dataset:
26
+ id: open-agent-leaderboard/results
27
+ task_id: swebench
28
+ value: 0.5204
29
+ source:
30
+ url: https://www.exgentic.ai
31
+ name: Open Agent Leaderboard
32
+ notes: 'agent: Claude Code'
33
+ - dataset:
34
+ id: open-agent-leaderboard/results
35
+ task_id: taubench_airline
36
+ value: 0.12
37
+ source:
38
+ url: https://www.exgentic.ai
39
+ name: Open Agent Leaderboard
40
+ notes: 'agent: Claude Code'
41
+ - dataset:
42
+ id: open-agent-leaderboard/results
43
+ task_id: taubench_retail
44
+ value: 0.03
45
+ source:
46
+ url: https://www.exgentic.ai
47
+ name: Open Agent Leaderboard
48
+ notes: 'agent: Claude Code'
49
+ - dataset:
50
+ id: open-agent-leaderboard/results
51
+ task_id: taubench_telecom
52
+ value: 0.0
53
+ source:
54
+ url: https://www.exgentic.ai
55
+ name: Open Agent Leaderboard
56
+ notes: 'agent: Claude Code'
.eval_results/open_agent_leaderboard_openai_solo.yaml ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: open-agent-leaderboard/results
3
+ task_id: overall
4
+ value: 0.2501
5
+ source:
6
+ url: https://www.exgentic.ai
7
+ name: Open Agent Leaderboard
8
+ notes: 'agent: OpenAI Solo'
9
+ - dataset:
10
+ id: open-agent-leaderboard/results
11
+ task_id: appworld
12
+ value: 0.08
13
+ source:
14
+ url: https://www.exgentic.ai
15
+ name: Open Agent Leaderboard
16
+ notes: 'agent: OpenAI Solo'
17
+ - dataset:
18
+ id: open-agent-leaderboard/results
19
+ task_id: browsecomp_plus
20
+ value: 0.35
21
+ source:
22
+ url: https://www.exgentic.ai
23
+ name: Open Agent Leaderboard
24
+ notes: 'agent: OpenAI Solo'
25
+ - dataset:
26
+ id: open-agent-leaderboard/results
27
+ task_id: swebench
28
+ value: 0.567
29
+ source:
30
+ url: https://www.exgentic.ai
31
+ name: Open Agent Leaderboard
32
+ notes: 'agent: OpenAI Solo'
33
+ - dataset:
34
+ id: open-agent-leaderboard/results
35
+ task_id: taubench_airline
36
+ value: 0.0
37
+ source:
38
+ url: https://www.exgentic.ai
39
+ name: Open Agent Leaderboard
40
+ notes: 'agent: OpenAI Solo'
41
+ - dataset:
42
+ id: open-agent-leaderboard/results
43
+ task_id: taubench_retail
44
+ value: 0.01
45
+ source:
46
+ url: https://www.exgentic.ai
47
+ name: Open Agent Leaderboard
48
+ notes: 'agent: OpenAI Solo'
49
+ - dataset:
50
+ id: open-agent-leaderboard/results
51
+ task_id: taubench_telecom
52
+ value: 0.0
53
+ source:
54
+ url: https://www.exgentic.ai
55
+ name: Open Agent Leaderboard
56
+ notes: 'agent: OpenAI Solo'
.eval_results/open_agent_leaderboard_smolagents_code.yaml ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: open-agent-leaderboard/results
3
+ task_id: overall
4
+ value: 0.42
5
+ source:
6
+ url: https://www.exgentic.ai
7
+ name: Open Agent Leaderboard
8
+ notes: 'agent: Smolagent'
9
+ - dataset:
10
+ id: open-agent-leaderboard/results
11
+ task_id: appworld
12
+ value: 0.11
13
+ source:
14
+ url: https://www.exgentic.ai
15
+ name: Open Agent Leaderboard
16
+ notes: 'agent: Smolagent'
17
+ - dataset:
18
+ id: open-agent-leaderboard/results
19
+ task_id: browsecomp_plus
20
+ value: 0.33
21
+ source:
22
+ url: https://www.exgentic.ai
23
+ name: Open Agent Leaderboard
24
+ notes: 'agent: Smolagent'
25
+ - dataset:
26
+ id: open-agent-leaderboard/results
27
+ task_id: swebench
28
+ value: 0.5761
29
+ source:
30
+ url: https://www.exgentic.ai
31
+ name: Open Agent Leaderboard
32
+ notes: 'agent: Smolagent'
33
+ - dataset:
34
+ id: open-agent-leaderboard/results
35
+ task_id: taubench_airline
36
+ value: 0.56
37
+ source:
38
+ url: https://www.exgentic.ai
39
+ name: Open Agent Leaderboard
40
+ notes: 'agent: Smolagent'
41
+ - dataset:
42
+ id: open-agent-leaderboard/results
43
+ task_id: taubench_retail
44
+ value: 0.7245
45
+ source:
46
+ url: https://www.exgentic.ai
47
+ name: Open Agent Leaderboard
48
+ notes: 'agent: Smolagent'
49
+ - dataset:
50
+ id: open-agent-leaderboard/results
51
+ task_id: taubench_telecom
52
+ value: 0.7071
53
+ source:
54
+ url: https://www.exgentic.ai
55
+ name: Open Agent Leaderboard
56
+ notes: 'agent: Smolagent'
.eval_results/open_agent_leaderboard_tool_calling.yaml ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: open-agent-leaderboard/results
3
+ task_id: overall
4
+ value: 0.4251
5
+ source:
6
+ url: https://www.exgentic.ai
7
+ name: Open Agent Leaderboard
8
+ notes: 'agent: React'
9
+ - dataset:
10
+ id: open-agent-leaderboard/results
11
+ task_id: appworld
12
+ value: 0.09
13
+ source:
14
+ url: https://www.exgentic.ai
15
+ name: Open Agent Leaderboard
16
+ notes: 'agent: React'
17
+ - dataset:
18
+ id: open-agent-leaderboard/results
19
+ task_id: browsecomp_plus
20
+ value: 0.34
21
+ source:
22
+ url: https://www.exgentic.ai
23
+ name: Open Agent Leaderboard
24
+ notes: 'agent: React'
25
+ - dataset:
26
+ id: open-agent-leaderboard/results
27
+ task_id: swebench
28
+ value: 0.5714
29
+ source:
30
+ url: https://www.exgentic.ai
31
+ name: Open Agent Leaderboard
32
+ notes: 'agent: React'
33
+ - dataset:
34
+ id: open-agent-leaderboard/results
35
+ task_id: taubench_airline
36
+ value: 0.62
37
+ source:
38
+ url: https://www.exgentic.ai
39
+ name: Open Agent Leaderboard
40
+ notes: 'agent: React'
41
+ - dataset:
42
+ id: open-agent-leaderboard/results
43
+ task_id: taubench_retail
44
+ value: 0.6465
45
+ source:
46
+ url: https://www.exgentic.ai
47
+ name: Open Agent Leaderboard
48
+ notes: 'agent: React'
49
+ - dataset:
50
+ id: open-agent-leaderboard/results
51
+ task_id: taubench_telecom
52
+ value: 0.83
53
+ source:
54
+ url: https://www.exgentic.ai
55
+ name: Open Agent Leaderboard
56
+ notes: 'agent: React'
.eval_results/open_agent_leaderboard_tool_calling_with_shortlisting.yaml ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: open-agent-leaderboard/results
3
+ task_id: overall
4
+ value: 0.4276
5
+ source:
6
+ url: https://www.exgentic.ai
7
+ name: Open Agent Leaderboard
8
+ notes: 'agent: React + Shortlisting'
9
+ - dataset:
10
+ id: open-agent-leaderboard/results
11
+ task_id: appworld
12
+ value: 0.1
13
+ source:
14
+ url: https://www.exgentic.ai
15
+ name: Open Agent Leaderboard
16
+ notes: 'agent: React + Shortlisting'
17
+ - dataset:
18
+ id: open-agent-leaderboard/results
19
+ task_id: browsecomp_plus
20
+ value: 0.34
21
+ source:
22
+ url: https://www.exgentic.ai
23
+ name: Open Agent Leaderboard
24
+ notes: 'agent: React + Shortlisting'
25
+ - dataset:
26
+ id: open-agent-leaderboard/results
27
+ task_id: swebench
28
+ value: 0.5714
29
+ source:
30
+ url: https://www.exgentic.ai
31
+ name: Open Agent Leaderboard
32
+ notes: 'agent: React + Shortlisting'
33
+ - dataset:
34
+ id: open-agent-leaderboard/results
35
+ task_id: taubench_airline
36
+ value: 0.62
37
+ source:
38
+ url: https://www.exgentic.ai
39
+ name: Open Agent Leaderboard
40
+ notes: 'agent: React + Shortlisting'
41
+ - dataset:
42
+ id: open-agent-leaderboard/results
43
+ task_id: taubench_retail
44
+ value: 0.6465
45
+ source:
46
+ url: https://www.exgentic.ai
47
+ name: Open Agent Leaderboard
48
+ notes: 'agent: React + Shortlisting'
49
+ - dataset:
50
+ id: open-agent-leaderboard/results
51
+ task_id: taubench_telecom
52
+ value: 0.83
53
+ source:
54
+ url: https://www.exgentic.ai
55
+ name: Open Agent Leaderboard
56
+ notes: 'agent: React + Shortlisting'