Skip to content

subgraph: Add template handlers to filters - #3277

Merged
mangas merged 1 commit into
masterfrom
filipe/dynamic-data-sources
Mar 4, 2022
Merged

mangas merged 1 commit into
masterfrom
filipe/dynamic-data-sources

Conversation

@mangas

@mangas mangas commented Feb 24, 2022 •

Copy link
Copy Markdown
Collaborator

Fixes #3257

@mangas
mangas force-pushed the filipe/dynamic-data-sources branch from 748913a to 40071e8 Compare February 24, 2022 21:39
@mangas
mangas marked this pull request as ready for review February 28, 2022 10:57
@mangas
mangas force-pushed the filipe/dynamic-data-sources branch 2 times, most recently from 45fec9d to ac6788d Compare February 28, 2022 11:09
Comment thread chain/ethereum/src/chain.rs Resolved
Comment thread chain/ethereum/src/adapter.rs Resolved

@evaporei evaporei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but if possible I would like @leoyvens or someone else to double check.

Changing the filters is quite sensitive, it seems to be correct though 🙂

@leoyvens leoyvens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If EthereumCallFilter::extend won't affect the wildcard filter, then it should assert that the filter is empty.

Comment thread graph/src/data/schema.rs Outdated
schemas: &HashMap<SchemaReference, Arc<Schema>>,
) -> Result<(), Vec<SchemaValidationError>> {
// Using this since std::array's .into_iter() doesn't work
// as expected (at least as of rustc 1.53)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a rebase artifact, this comment should be removed.

Comment thread chain/ethereum/src/chain.rs Outdated

/// Ideal number of triggers in a range. The range size will adapt to try to meet this.
static ref TARGET_TRIGGERS_PER_BLOCK_RANGE: u64 = env_var("GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE", 100);
static ref TARGET_TRIGGERS_PER_BLOCK_RANGE: u64 = env_var("GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE", 1000);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems safest to not change this value until we have enabled the overall change by default.

Comment thread chain/ethereum/src/adapter.rs Outdated
(3, HashSet::from_iter(vec![[3u8; 4]])),
),
]),
..Default::default()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to replace these ..Default with wildcard_matches: HashSet::new()

Comment thread chain/ethereum/src/adapter.rs Resolved
Comment thread chain/ethereum/src/adapter.rs Outdated
None if self.wildcard_matches.is_empty() => return false,
// There are no signatures but there are wildcards.
None => None,
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On EthereumCallFilter::matches, first the correct_input_size check should be moved above to where we check call.input.0.len() < 4 (@otaviopace we missed this in #3219).

And then I'd refactor the correct_fn logic into the arms of this match, it would be more readable.

@mangas
mangas force-pushed the filipe/dynamic-data-sources branch from ac6788d to a5cd2c2 Compare February 28, 2022 16:21
Comment thread chain/ethereum/src/adapter.rs Outdated
let block = EthereumBlockFilter::from_mapping(&data_source.mapping);
if !block.is_empty() {
self.block.extend(block);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extending with an empty block is a noop. If the if is meant as an optimization, then it can be implemented within extend.

Comment thread chain/ethereum/src/adapter.rs Resolved
@mangas
mangas force-pushed the filipe/dynamic-data-sources branch from 1ea6531 to dc0eb76 Compare March 1, 2022 11:17
@mangas
mangas requested a review from leoyvens March 1, 2022 11:43
@mangas
mangas force-pushed the filipe/dynamic-data-sources branch from 20f2e0d to 3b1a92b Compare March 1, 2022 13:24
Comment thread node/src/main.rs Outdated
let restart_on_dynamic_data_source =
!env::var_os("EXPERIMENTAL_DISABLE_RESTART_ON_DYNAMIC_DS")
.map(|_| true)
.unwrap_or(false);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of trying to give this change a more meaningful and shorter name. Perhaps "static filters"? This would mean renaming restart_on_dynamic_data_source to static_filters_enabled and flipping the conditionals since enabling static filters means restarts are disabled.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good!

@mangas
mangas force-pushed the filipe/dynamic-data-sources branch from 6e30907 to 076b495 Compare March 3, 2022 17:19
@mangas
mangas requested a review from leoyvens March 3, 2022 17:20
@mangas
mangas force-pushed the filipe/dynamic-data-sources branch from b408048 to 830315b Compare March 4, 2022 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve block stream pipelining with dynamic data sources

3 participants