ICCRL Competition Rules and Regulations
Version 4.2 | Last Updated: January 2025
These rules govern all ICCRL competitions and rating activities. All participants are expected to be familiar with these regulations before submitting engines for testing.
Table of Contents
1. Standard Testing Conditions
1.1 Hardware Specifications
| CPU: | Intel Core i7-13700K @ 3.4 GHz base (5.4 GHz boost) |
| Cores Allocated: | 4 cores per engine (8 threads with hyperthreading) |
| RAM: | 16 GB DDR4-3200 (1024 MB hash per engine) |
| Storage: | NVMe SSD (Samsung 980 PRO 1TB) |
| Operating System: | Ubuntu 22.04 LTS 64-bit |
1.2 Time Controls
- Primary Rating List: 40 moves in 15 minutes, then 15 minutes for the rest of the game (40/15+15)
- Blitz Rating List: 3 minutes + 2 seconds increment per move (3+2)
- Rapid Rating List: 15 minutes + 10 seconds increment per move (15+10)
- Fischer Random (Chess960): 40 moves in 15 minutes, repeating (40/15)
1.3 Software Configuration
| GUI/Manager: | Cutechess-cli v1.3.1 |
| Opening Book: | ICCRL_openings_2025.bin (8-move variety, balanced) |
| Endgame Tablebases: | Syzygy 6-man (complete) + 7-man (selected, WDL + DTZ) |
| Contempt: | Disabled (set to 0 for both engines) |
| Pondering: | Disabled |
1.4 GPU Testing (Neural Network Events)
For specialized neural network events, additional GPU resources are provided:
- NVIDIA RTX 4090 (24GB VRAM)
- CUDA 12.1 / cuDNN 8.9
- One GPU per engine (no sharing)
- Separate queue from standard CPU testing
2. Engine Requirements
2.1 Eligibility Criteria
All engines must meet these requirements:
- Support UCI (Universal Chess Interface) protocol
- Be freely available OR authorized by developer for testing
- Function correctly on Linux x64 platform
- Produce no illegal moves during testing
- Not crash or hang during normal operation
- Be deterministic (no intentional randomness in move selection)
Note: Neural network engines must include all required network weight files and dependency libraries. Engines requiring internet connectivity are not eligible.
2.2 Prohibited Engines
The following types of engines are not eligible for ICCRL testing:
- Direct clones or trivially modified versions of existing engines without permission
- Engines requiring internet/cloud connectivity during play
- Commercial engines without explicit developer authorization
- Engines with known bugs causing illegal moves or frequent crashes
- Engines using pre-computed perfect play databases beyond standard tablebases
- Engines specifically designed to exploit known weaknesses of particular opponents
2.3 Version Management
New Version Testing: New versions must demonstrate significant changes to warrant re-testing. Minor bug fixes without strength impact may not be prioritized.
Development Versions: Development/unstable versions are tested at developer request but flagged appropriately in the rating list.
Version Identification: Engines must properly identify themselves with unique version numbers via the UCI id command.
2.4 Source Code and Licensing
While source code availability is preferred, it is not mandatory for testing. However, engines derived from open-source projects must:
- Properly attribute the original work
- Comply with the original license terms
- Clearly document modifications made
- Not misrepresent the engine's origins
3. Competition Format
3.1 Game Rules and Adjudication
| Rule | Specification |
|---|---|
| Pairing System | Each engine plays both colors against every opponent |
| Minimum Games | 1000 games per engine for stable rating |
| Draw Adjudication | 8-move rule with evaluation ≤ 0.10 pawns (both sides) |
| Win Adjudication | 4-move rule with evaluation ≥ 10.00 pawns |
| Maximum Moves | 350 moves (adjudicated as draw if reached) |
| 3-fold Repetition | Automatic draw by GUI when detected |
| 50-move Rule | Automatic draw when applicable |
3.2 Opening Book Policy
Uniformity: All engines use the identical opening book provided by ICCRL. Engine-specific opening books are disabled.
Book Exits: The standard book typically exits between moves 8-12, ensuring sufficient variety while avoiding theoretical endgames.
Position Selection: Openings are selected for:
- Diversity of pawn structures and piece configurations
- Practical relevance (not fringe or refuted lines)
- Balanced evaluation (not heavily favoring one side)
- Sufficient complexity to test engine strength
3.3 Rating Calculation
System: Elo rating system (standard chess implementation)
K-factor: 10 (lower K-factor for stability with large game samples)
Initial Rating: 2000 for all new engines
Provisional Period: First 50 games (rating marked as provisional)
Update Frequency: Monthly (15th of each month)
Error Margins: Calculated at 95% confidence interval based on game sample
3.4 Game Logs and Transparency
- All games are logged in PGN format with full metadata
- Engine output and thinking time recorded for review
- Game archives publicly available after rating publication
- Statistical summaries provided for each testing cycle
4. Conduct and Fair Play
4.1 Developer Responsibilities
Developers must:
- Respect open-source licenses and properly attribute derived work
- Not submit multiple entries that are essentially identical
- Provide accurate engine identification and version information
- Respond to configuration questions within reasonable timeframes
- Report any known bugs or limitations that might affect testing
4.2 Testing Integrity
Isolation: Each engine is tested independently without knowledge of opponents' identities or playing styles.
Consistency: All engines are tested under identical conditions during the same time period to ensure hardware and software consistency.
Error Handling: Hardware failures, software crashes, or system issues result in game replays, not forfeitures.
Anomaly Investigation: Unusual patterns or results are investigated before publication. Developers are contacted if issues are suspected.
4.3 Prohibited Conduct
The following actions may result in disqualification:
- Intentional misidentification of engine origin or authorship
- Attempting to exploit testing infrastructure vulnerabilities
- Submitting engines designed to cheat or exploit specific opponents
- Harassment or unprofessional conduct toward ICCRL staff or other developers
- Deliberately submitting buggy or malicious code
5. Appeals and Disputes
5.1 Appeal Process
- Submission Window: Appeals must be submitted within 30 days of rating publication
- Written Request: Email appeals@iccrl.org with detailed explanation and evidence
- Committee Review: ICCRL technical committee reviews within 14 days
- Response: Developer receives written response with decision and rationale
- Re-testing: If appeal is upheld, games may be replayed or rating corrected
5.2 Valid Grounds for Appeal
- Incorrect engine configuration (wrong version, parameters, etc.)
- Evidence of hardware/software malfunction affecting results
- Statistical anomalies suggesting testing errors
- Violation of stated testing conditions
- Technical issues causing crashes or time losses
5.3 Rating Corrections
If errors are confirmed, ICCRL will issue corrections retroactively:
- Ratings adjusted based on corrected game results
- Public announcement of correction with explanation
- Historical records updated to reflect accurate data
- Affected developers notified of all changes
6. Submission Guidelines
6.1 Required Information
All engine submissions must include:
- Engine binary: Linux x64 executable (UCI protocol)
- Engine name: Official name and version number
- Developer information: Name and email contact
- License/Authorization: Confirmation of permission to test
- Dependencies: Any required libraries or network files
- Configuration notes: Recommended UCI parameters or special requirements
- Source/Origin: Link to source code or official download page (if applicable)
6.2 Submission Process
- Email submission to submissions@iccrl.org
- Receive acknowledgment within 3 business days
- Engine added to testing queue (typical wait: 2-4 weeks)
- Preliminary testing to verify functionality
- Full rating games commence (1000+ games over 4-6 weeks)
- Results reviewed and published on monthly update
6.3 Contact Information
| Engine Submissions: | submissions@iccrl.org |
| Rules Inquiries: | rules@iccrl.org |
| Technical Issues: | technical@iccrl.org |
| Appeals: | appeals@iccrl.org |
| General Contact: | info@iccrl.org |
7. Performance Metrics and Analysis
7.1 Standard Performance Indicators
In addition to Elo ratings, ICCRL tracks comprehensive performance metrics for each engine:
| Metric | Description |
|---|---|
| Win Rate | Percentage of games won (excludes draws) |
| Draw Rate | Percentage of games ending in draws |
| Performance Rating | Calculated rating based on opponents' strength and results |
| Average Game Length | Mean number of moves in completed games |
| Time Management | Average time remaining at game conclusion |
| Endgame Conversion | Success rate in converting won positions (eval > +2.00) |
7.2 Head-to-Head Records
ICCRL maintains detailed head-to-head statistics between all tested engines, including:
- Win/Draw/Loss records for each pairing
- Color-specific statistics (White vs Black performance)
- Opening-specific performance analysis
- Historical trends across multiple testing cycles
7.3 Statistical Significance
All rating changes and performance claims are evaluated for statistical significance:
- Minimum Games: 50 games required for provisional ratings, 1000+ for stable ratings
- Confidence Intervals: 95% confidence intervals calculated for all ratings
- Error Margins: Published alongside ratings (e.g., "3500 ± 15")
- LOS (Likelihood of Superiority): Calculated for engine comparisons
8. Anti-Cheating and Integrity Measures
8.1 Clone Detection
ICCRL employs multiple methods to detect engine clones and derivatives:
- Move Similarity Analysis: Statistical comparison of move choices in identical positions
- Evaluation Correlation: Comparing position evaluations across engines
- Code Analysis: Binary comparison and string analysis (when available)
- Behavioral Fingerprinting: Time usage patterns and search characteristics
- Community Reporting: Developer and community flagging of suspicious entries
8.2 Engine Identity Verification
Verification Requirements:
- Engines must properly identify via UCI "id name" and "id author" commands
- Version numbers must be unique and incrementally versioned
- Developer email must be verifiable and responsive
- Substantial code modifications required for derivatives (minimum 30% new code)
- Clear attribution required for all borrowed code or concepts
8.3 Sandboxing and Security
All engine testing occurs in controlled, isolated environments:
- Engines run in sandboxed Docker containers with limited permissions
- Network access completely disabled during testing
- File system access restricted to designated directories
- Process monitoring for unusual system calls or behavior
- Resource limits enforced (CPU, memory, disk I/O)
8.4 Disciplinary Actions
| Violation | Action |
|---|---|
| Minor misidentification (accidental) | Warning + correction required |
| Unattributed derivative work | Rating removal + 6-month suspension |
| Intentional clone submission | Permanent ban from ICCRL |
| Malicious code | Permanent ban + legal action |
| Repeated rule violations | Escalating suspensions up to permanent ban |
9. Data Privacy and Retention
9.1 Developer Information
Collection and Usage:
- Developer names and email addresses collected for communication purposes only
- Contact information never sold, shared, or used for marketing
- Developer names published alongside engine listings (unless anonymity requested)
- Email addresses kept private and used only for ICCRL-related correspondence
9.2 Game Data and Archival
Public Data:
- All game PGNs publicly available after rating publication
- Complete move histories, evaluations, and time stamps included
- Statistical summaries available for download
- Historical data retained indefinitely for research purposes
Private Data:
- Engine binaries stored securely and not redistributed
- Developer communications kept confidential
- Internal testing logs retained for 2 years (appeals window)
9.3 Data Retention Schedule
| Data Type | Retention Period |
|---|---|
| Game PGNs | Permanent (public archive) |
| Rating histories | Permanent (public record) |
| Engine binaries | 2 years minimum (for re-testing if needed) |
| Developer contact info | While engine listed + 1 year |
| Email correspondence | 3 years |
| Internal testing logs | 2 years |
9.4 Right to Removal
Engine developers have the right to request removal of their engines from active testing:
- Active Rating Removal: Engine removed from current list within 30 days of request
- Historical Data: Past competition results remain in archive for historical accuracy
- Personal Data: Developer contact information purged within 90 days
- Reinstatement: Removed engines can be resubmitted following standard procedures
10. Special Circumstances and Exceptions
10.1 Hardware Limitations
Engines with special hardware requirements may receive modified testing conditions:
- GPU Engines: Tested in separate GPU-enabled queue with NVIDIA RTX 4090
- High Memory Engines: Exceptions granted for engines requiring > 1GB hash (up to 4GB)
- Multi-core Specialists: Additional testing cycles with varied core counts (2, 4, 8 cores)
- NUMA Aware Engines: Testing on NUMA-enabled systems when requested
Note: Special requirements must be disclosed at submission and may result in longer testing queues.
10.2 Platform-Specific Engines
While ICCRL primarily tests Linux x64 engines, limited testing is available for:
- Windows Binaries: Tested via Wine compatibility layer (best effort)
- ARM Engines: Separate ARM64 rating list maintained (quarterly updates)
- Web Engines (WASM): Experimental testing in browser environment
Important: Non-Linux engines may experience compatibility issues. Developers are encouraged to provide native Linux builds when possible.
10.3 Educational and Research Engines
Special provisions for non-competitive engines used for education or research:
- Engines can be flagged as "Educational" or "Research" in listings
- Reduced minimum game requirements (250 games vs 1000)
- Ratings published but clearly marked as experimental
- Priority given to novel or academically interesting approaches
- Exemptions from clone-detection rules if properly documented
10.4 Legacy Engine Support
ICCRL maintains support for historical engines with special considerations:
- Older Protocols: Limited support for WinBoard/XBoard protocol engines
- Compatibility Mode: Older engines tested with period-appropriate opening books
- Historical Ratings: Separate "Legacy" rating list for engines no longer maintained
- Preservation: Archived binaries maintained for chess engine history research
Rule Changes and Updates
ICCRL rules are subject to periodic review and updates to reflect evolving technology and community feedback.