EIP-8246 Test Checklist¶

Depending on the changes introduced by an EIP, the following template is the minimum baseline to guarantee test coverage of the Execution Layer features.

Checklist Progress Tracker¶

General¶

Code coverage¶

Fuzzing¶

Fuzzing is recommended to be performed on EIPs that introduce new cryptography primitives.

See holiman/goevmlab for an example of a fuzzing framework for the EVM.

New Opcode¶

The EIP introduces one or more new opcodes to the EVM.

Test Vectors¶

Memory expansion¶

If the opcode execution can expand the memory size, either by writing to memory or reading from an offset that exceeds current memory, or interaction of both parameters (size of zero should never result in memory expansion, regardless of offset value).

Stack¶

Stack Overflow¶

If the opcode pushes one or more items to the stack, and the opcode pushes more elements than it pops, verify that the opcode execution results in exceptional abort when pushing elements to the stack would result in the stack having more than 1024 elements.

Stack Underflow¶

If the opcode pops one or more items to the stack, or it has a minimum stack height of one or more (e.g. DUPN requires a minimum amount of elements in the stack even when it does not pop any element from it), verify that the opcode execution results in exceptional abort then stack has 1 less item than the minimum stack height expected.

Stack Complex Operations¶

If opcode performs stack operations more complex than simple pop/push (e.g. the opcode has a data portion that specifies a variable to access a specific stack element), perform the following test combinations.

Stack Manipulation With Data Portion Variables¶

If the opcode contains variables in its data portion, for each variable n of the opcode that accesses the nth stack item, test n being.

Execution context¶

Evaluate opcode's behavior in different execution contexts.

CALL¶

Verify opcode operation in a subcall frame originated from a CALL opcode.

STATICCALL¶

Verify opcode operation in a subcall frame originated from a STATICCALL opcode.

DELEGATECALL¶

Verify opcode operation in a subcall frame originated from a DELEGATECALL opcode.

CALLCODE¶

Verify opcode operation in a subcall frame originated from a CALLCODE opcode.

Initcode¶

Verify opcode behaves as expected when running during the initcode phase of contract creation.

Set-code Delegated Account¶

Verify opcode operations are applied to the set-code account and do not affect the address that is the target of the delegation.

Transaction Context¶

If opcode changes behavior depending on particular transaction properties, test using multiple values for each property.

Block Context¶

If opcode changes behavior depending on particular block properties, test using multiple values for each property.

Return data¶

Verify proper return data buffer overwriting if the opcode is meant to interact with it, otherwise verify that the return data buffer is unaffected.

Gas Usage¶

Normal Operation¶

Verify gas usage affectation of each stack argument or memory input consumed by the opcode.

Memory Expansion¶

Verify that the memory expansion correctly follows the gas calculation.

Extra Gas¶

Verify that attempting to execute the opcode when gas available is 1 more than the required gas results in exceptional abort.

Out-Of-Gas¶

Verify that attempting to execute the opcode when gas available is 1 less than the required gas results in exceptional abort.

Order-of-operations¶

If the opcode requires different gas stipends for other operations (e.g. contract creation, cold/warm account access), create one case for each operation (ideally independent of each other) and the listed cases for each.

Terminating opcode¶

If an opcode is terminating, meaning it results in the current call context to end execution.

Aborting/Reverting opcode¶

If the terminating opcode is meant to rollback the executing call frame, verify the listed events are properly rolled back.

Out-of-bounds checks¶

If the opcode has out-of-bounds conditions in its parameters/inputs.

Exceptional Abort¶

If the opcode has conditions, either inputs or execution context states, that should cause exceptional abort and are different than out-of-gas or stack overflow or underflow.

Data portion¶

If an opcode has a data portion, meaning the N bytes following the opcode in the contract bytecode are skipped from execution.

Contract creation¶

If the opcode execution results in the creation of a new contract in the state.

Correct Creation¶

Verify contract is created at the expected address given multiple inputs.

Creation Failure¶

The contract creation fails given the listed conditions.

Recursive Contract Creation¶

Opcode is used to attempt to recreate a contract that is currently mid-creation by a previous call of the same opcode.

Fork transition¶

Framework Changes¶

• Add opcode to packages/testing/src/execution_testing/vm/opcode.py
• Add opcode to relevant methods in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py

New Precompile¶

The EIP introduces one or more new precompiles.

Test Vectors¶

Call contexts¶

Evaluate precompile behavior when called using different call opcodes or called from different execution contexts.

CALL¶

Verify precompile operation when called using the CALL opcode.

DELEGATECALL¶

Verify precompile operation when called using the DELEGATECALL opcode.

STATICCALL¶

Verify precompile operation when called using the STATICCALL opcode.

If the precompile is stateful, meaning calling it affects its storage, this call must result in exceptional abort.

CALLCODE¶

Verify precompile operation when called using the CALLCODE opcode.

Transaction Entry-point¶

Verify precompile behavior when it's used as tx.to.

Initcode call¶

Verify calling the opcode during initcode execution of a new contract.

Precompile as Set-code Delegated Address¶

Test setting the precompile as a set-code delegated address, and verify no precompile logic is executed.

Inputs¶

Value Transfer¶

Minimum Fee Precompile¶

If the precompile requires a minimum value (fee) to execute, either constant or depending on a formula.

No-Fee Precompile¶

If the precompile does not require any minimum value (fee) to execute.

Out-of-bounds checks¶

If the precompile has out-of-bounds conditions in its inputs.

Input Lengths¶

Verify different combinations of input lengths to the precompile, ensuring the correct minimum fee (if any) is provided.

Zero-length Input¶

Regardless of the input requirements for the precompile.

Static Required Input Length¶

If the precompile has a static required input length.

Dynamic Required Input Length¶

If the precompile has a variable required input-length based on a formula, test all listed scenarios given different input lengths.

Gas usage¶

Constant Gas Cost¶

If the precompile always charges the same gas cost regardless of input conditions.

Variable Gas Cost¶

If the precompile charges variable gas cost given input conditions, test all listed scenarios using multiple different valid inputs.

Excessive Gas¶

Verify spending all block gas in calls to the precompile (Use Environment().gas_limit as reference max amount).

Fork transition¶

Pre-fork Block Call¶

Verify that calling the precompile before its activation fork results in a valid call, even for inputs that are expected to be invalid for the precompile, or a zero-gas call.

Cold/Warm Precompile Address State¶

Verify the cold/warm state of the precompile address depending on the fork activation.

Framework Changes¶

• Add precompile address to relevant methods in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py

Removed Precompile¶

The EIP removes one or more precompiles from the existing list of precompiles.

Test Vectors¶

Fork transition¶

Precompile Operation¶

Verify that the precompile remains operational up until the last block before the fork is active, and behaves as an account with empty code afterwards.

Cold/Warm Precompile Address State¶

Verify the cold/warm state of the precompile address depending on the fork activation.

Framework Changes¶

• Remove the precompile address from relevant methods in the fork where the EIP is removed in packages/testing/src/execution_testing/forks/forks/forks.py

New System Contract¶

Test Vectors¶

Call contexts¶

Evaluate precompile behavior when called using different call opcodes or called from different execution contexts.

CALL¶

Verify system contract operation when called using the CALL opcode.

DELEGATECALL¶

Verify system contract operation when called using the DELEGATECALL opcode.

STATICCALL¶

Verify system contract operation when called using the STATICCALL opcode.

If the system contract is stateful, meaning calling it affects its storage, this call must result in exceptional abort.

CALLCODE¶

Verify system contract operation when called using the CALLCODE opcode.

Transaction Entry-point¶

Verify system contract behavior when it's used as tx.to.

Initcode call¶

Verify calling the opcode during initcode execution of a new contract.

System Contract as Set-code Delegated Address¶

Test setting the system contract as a set-code delegated address, and verify no system contract side-effects are triggered, even if the actual system contract logic is executed.

If the system contract requires specific storage pre-conditions to be set for proper execution (e.g. if the contract contains a safety mechanism that prevents it from executing prior to the fork activation), ensure the delegated account has the proper values in the storage to guarantee the correct logic is executed.

Inputs¶

Value Transfer¶

Minimum Fee System Contract¶

If the system contract requires a minimum value (fee) to execute, either constant or depending on a formula.

No-Fee System Contract¶

If the system contract does not require any minimum value (fee) to execute.

Out-of-bounds checks¶

If the system contract has out-of-bounds conditions in its inputs.

Input Lengths¶

Verify different combinations of input lengths to the system contract, ensuring the correct minimum fee (if any) is provided.

Zero-length Input¶

Regardless of the input requirements for the system contract.

Static Required Input Length¶

If the system contract has a static required input length.

Dynamic Required Input Length¶

If the system contract has a variable required input-length based on a formula, test all listed scenarios given different input lengths.

Gas usage¶

Constant Gas Cost¶

If the system contract always charges the same gas cost regardless of input conditions.

Variable Gas Cost¶

If the system contract charges variable gas cost given input conditions, test all listed scenarios using multiple different valid inputs.

Excessive Gas Cases¶

Excessive Gas Usage During Block Execution¶

Verify spending all block gas in calls to system contract (Use Environment().gas_limit as reference max amount).

Excessive Gas Usage During System Call¶

If possible, produce a scenario where, given all transactions executed within the block result in the execution of the contract by the system address would result in excessive gas usage.

System Contract Deployment¶

Contract Variations¶

Verify execution of the different variations of the contract for different networks (if any) results in the expected behavior.

Contract Substitution¶

Substitute the default system contract with a mock contract to modify its behavior when called by the system address (at the end of the block execution).

Fork transition¶

Verify calling the system contract before its activation fork results in correct behavior (depends on the system contract implementation).

Framework Changes¶

• Add system contract address to relevant methods in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py
• Add system contract bytecode to the returned value of pre_allocation_blockchain in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py

New Transaction Type¶

Test Vectors¶

Intrinsic Validity¶

Verify the transaction (and the block it is included in) is valid or invalid given the intrinsic validity of the new transaction type, depending on each test scenario.

For each new field that affects the intrinsic gas cost of the transaction verify all listed scenarios.

Gas Limit¶

Note: Data floor gas cost affects the intrinsic validity of all transaction types, so it must be taken into account when designing all test scenarios.

Gas Fee¶

Chain ID¶

Nonce¶

To¶

Value¶

Data¶

Signature¶

Verify the transaction is correctly rejected if it contains an invalid signature.

V¶

R¶

S¶

Transaction Attributes Readable From EVM¶

Transaction-Scoped Persistent Values¶

Verify values or variables that are persistent through the execution of the transaction (e.g. transient storage, warm/cold accounts).

Encoding (RLP, SSZ)¶

Verify correct transaction rejection in all listed scenarios.

Field Sizes¶

Verify all listed scenarios for each transaction field.

Fields of List Type¶

Verify for each transaction field that is of type list.

Extra/Missing Fields¶

Serialization Corruption¶

Serializable Fields¶

Verify for each serializable field, all previous tests plus following listed scenario.

Out-of-bounds checks¶

Verify if the transaction has out-of-bounds conditions in its fields and verify.

Contract creation¶

Verify that the transaction can create new contracts if the transaction type supports it.

Sender account modifications¶

Block Level Interactions¶

Single Transaction In Block¶

Verify a block where the new transaction type is the sole transaction contained in the block.

Two Transactions In Block¶

Verify a block where the new transaction type is the last transaction contained in a block with two transactions.

EIP-7825¶

Verify a transaction of the new type is rejected if its gas limit exceeds the EIP-7825 gas limit for the current fork.

Mixed transactions¶

Verify a block with all transactions types including the new type is executed correctly.

Fork transition¶

Verify that a block prior to fork activation where the new transaction type is introduced and containing the new transaction type is invalid.

RPC Tests¶

• *Verify eth_estimateGas behavior for different valid combinations of the new transaction type.
• transaction_type/test/rpc/send_raw | Verify eth_sendRawTransaction using execute.

*Tests must be added to execution-apis repository.

Framework Changes¶

• Modify transaction_intrinsic_cost_calculator in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py, adding the appropriate new fields that the transaction introduced and the logic to the intrinsic gas cost calculation, if any.
• Add the transaction type number to tx_types response in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py (If applicable add also to contract_creating_tx_types).

New Block Header Field¶

Test Vectors¶

Genesis value¶

Verify, if possible, that the value can be set at genesis if the network starting fork is the activation fork, and that clients can consume such genesis.

Value behavior¶

Verify, given multiple initial values, that a block is accepted or rejected depending on the correct expected value for the current block.

Fork transition¶

Framework Changes¶

• Add the new header field to the relevant objects:
  • execution_testing.fixtures.FixtureHeader.
  • execution_testing.fixtures.FixtureExecutionPayload.
  • execution_testing.specs.Header.
• Add the appropriate header_*_required fork method to BaseFork in execution_testing.forks.

New Block Body Field¶

Test Vectors¶

Value behavior¶

Verify, given multiple initial values, that a block is accepted or rejected depending on the correct expected value for the current block.

Fork transition¶

Framework Changes¶

• Add the new body field to the relevant objects.
  • execution_testing.fixtures.FixtureBlockBase.
  • execution_testing.fixtures.FixtureEngineNewPayload.
  • execution_testing.specs.Block.
• Modify execution_testing.specs.BlockchainTest filling behavior to account for the new block field.

Gas Cost Changes¶

Test Vectors¶

Gas Usage¶

Measure and store the gas usage during the operations affected by the gas cost changes and verify the updated behavior.

Out-of-gas¶

Verify the operations affected by the gas cost changes can run out-of-gas with the updated limits.

Fork transition¶

Verify gas costs are updated at the fork transition boundary.

Framework Changes¶

• Modify transaction_intrinsic_cost_calculator in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py if the EIP affects intrinsic gas cost calculation.
• Modify transaction_data_floor_cost_calculator in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py if the EIP affects calldata floor cost.
• Modify memory_expansion_gas_calculator in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py if the EIP affects memory expansion gas cost calculation.
• Modify gas_costs in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py if the EIP affects specific opcode gas costs.

Gas Refunds Changes¶

Test Vectors¶

Refund calculation¶

Verify that the refund does not exceed gas_used // MAX_REFUND_QUOTIENT (MAX_REFUND_QUOTIENT==5 in EIP-3529) in the following scenarios.

Exceptional Abort¶

Cross-Functional Test¶

Verify the following tests are updated to support the new type of refunds.

Framework Changes¶

N/A

Blob Count Changes¶

Test Vectors¶

Existing Test Updates¶

Verify tests in tests/cancun/eip4844_blobs were correctly and automatically updated to take into account the new blob count values at the new fork activation block.

Framework Changes¶

• Modify blob_base_fee_update_fraction, target_blobs_per_block, max_blobs_per_block in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py if the EIP affects any of the values returned by each function.

New Execution Layer Request¶

Test Vectors¶

Cross-Request-Type Interaction¶

Framework Changes¶

• Increment max_request_type in the fork where the EIP is introduced in packages/testing/src/execution_testing/forks/forks/forks.py to the new maximum request type number after the EIP is activated.

New Transaction-Validity Constraint¶

Test Vectors¶

Fork transition¶

Note: All test cases must use off-by-one values to ensure proper boundary condition verification.

Framework Changes¶

• Introduce the validity constraint as a fork method that returns:
  • None for forks before its activation.
  • A non-None value starting from the fork where the constraint becomes active.

Modified Transaction-Validity Constraint¶

Test Vectors¶

Fork transition¶

Note: All test cases must use off-by-one values to ensure proper boundary condition verification.

Framework Changes¶

• Update the validity constraint as a fork method that returns the updated value starting from the fork where the constraint changes.

Block-Level Validation Constraint¶

The EIP introduces a new constraint that applies at the block level (e.g., block RLP size limits, block validation rules).

Test Vectors¶

Boundary Conditions¶

Verify block acceptance/rejection at constraint boundaries using off-by-one values.

Content Variations¶

Verify the constraint behaves correctly with different block contents that may affect the constraint calculation.

Fork Transition¶

Note: All test cases must use off-by-one values to ensure proper boundary condition verification.

Framework Changes¶

• Introduce the constraint as a fork method that returns:
  • None for forks before its activation.
  • A non-None value starting from the fork where the constraint becomes active.