Skip to content

fix(io): pass GCS token expiry to gcsfs - #4089

Merged
kevinjqliu merged 1 commit into
apache:mainfrom
amdubovoi:fix/gcs-token-expiry
Nov 21, 2026
Merged

kevinjqliu merged 1 commit into
apache:mainfrom
amdubovoi:fix/gcs-token-expiry

Conversation

@amdubovoi

Copy link
Copy Markdown
Contributor

Rationale for this change

FsspecFileIO passes the GCS token to gcsfs without its expiry. gcsfs 2026.2.0 and later then look up the expiry at tokeninfo, which fails for down-scoped tokens that REST catalogs vend. Reads fail with Provided token is either not valid, or expired.

A token with gcs.oauth2.token-expires-at now goes to gcsfs as google.oauth2.credentials.Credentials with that expiry.

Are these changes tested?

Yes, with unit tests and against a REST catalog that vends down-scoped GCS tokens.

Are there any user-facing changes?

Yes. Vended GCS credentials work again with gcsfs 2026.2.0 and later.

@amdubovoi

Copy link
Copy Markdown
Contributor Author

@rambleraptor @Fokko is it possible to get a review here, or have I done anything sidestepping the process?

@amdubovoi
amdubovoi force-pushed the fix/gcs-token-expiry branch 2 times, most recently from 1efa819 to 12b4ccb Compare October 28, 2026 09:11
@amdubovoi

Copy link
Copy Markdown
Contributor Author

@kevinjqliu May I ask you to take a look since you've touched gcsfs stuff previously? And sorry for pinging once again, kinda blocked by this 😢

@amdubovoi
amdubovoi force-pushed the fix/gcs-token-expiry branch from 12b4ccb to c8ae719 Compare November 21, 2026 09:39

@kevinjqliu kevinjqliu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread pyiceberg/io/fsspec.py

token: str | Credentials | None = properties.get(GCS_TOKEN)
if token and (expires_at := properties.get(GCS_TOKEN_EXPIRES_AT_MS)):
token = Credentials(token, expiry=millis_to_datetime(int(expires_at)))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kevinjqliu
kevinjqliu added this pull request to the merge queue Nov 21, 2026
Merged via the queue into apache:main with commit ee86b4b Nov 21, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants