Repository navigation
Dont list directorymarkers in iterdir - #536
Conversation
|
Perhaps we should also check for |
|
It seems, the way info is implemented in s3fs, ContentType is currently not propagated for these directory markers. |
|
If you could explicitly confirm that the PR as-is fixes your issue I'd be happy to merge |
So the artificial marker object breaks |
s3fs doesn't let you do this like the aws console would because it strips the trailing slash |
|
Hmmm, seems like yet another edge case. Okay I'll see if this can be handled in a better way. |
On object stores an object key can also be the prefix of other objects. s3fs then lists the name twice (as file and as directory), and once the parent listing is cached, isdir() returns False for that name, so iterdir() yielded duplicates and raised NotADirectoryError when listing the colliding directory (see fsspec/s3fs#999). iterdir() now skips duplicate names and re-checks isdir() once after invalidating the filesystem cache before raising NotADirectoryError.
The directory marker check only matched s3 listings of non-root prefixes (base paths without a trailing separator, with s3fs specific info keys). Listing entries whose name equals the listed directory are never children, so skip them for all filesystems. This also fixes iterdir() yielding a bogus 's3://bucket/bucket' child for buckets with keys starting with '/' (fsspec#144).
b5d91d2 to
910c969
Compare

Should close #226