Conversation
This comment was marked as outdated.
This comment was marked as outdated.
| #[derive(Deserialize, Debug)] | ||
| pub struct OrganizationDetails { | ||
| pub id: String, | ||
| pub links: OrganizationLinks, |
There was a problem hiding this comment.
This endpoint has a bunch more information but what we really care about is the id and links.
The region_url contains the regional API url, e.g. us.sentry.io. This way we can hit that endpoint directly instead of going through control silo (sentry.io).
If I understand correctly I think this information might already be somehow embedded in org tokens, but not in personal tokens. Or maybe I'm confusing this with something else.
| use anyhow::{Context, Result}; | ||
|
|
||
| /// Given an org and project slugs or IDs, returns the IDs of both. | ||
| pub fn get_org_project_id(api: impl AsRef<Api>, org: &str, project: &str) -> Result<(u64, u64)> { |
There was a problem hiding this comment.
As far as I know, for all commands a user can provide --project and --org as slugs or IDs.
These utils are needed to get the corresponding IDs, so that objects from the same org/proj all have the same paths in objectstore regardless if the user passed in slugs or IDs.
There was a problem hiding this comment.
It might be weird to take in Api and it might be possible that these functions should live somewhere else, IDK.
They certainly don't belong to the Api struct as its methods seem to all map 1to1 to Sentry API calls.
| lazy_static = "1.4.0" | ||
| libc = "0.2.139" | ||
| log = { version = "0.4.17", features = ["std"] } | ||
| objectstore-client = { git = "https://github.com/getsentry/objectstore.git", branch = "lcian/feat/rust-batch-client" } |
There was a problem hiding this comment.
This indirectly adds a dependency to reqwest and we need to double check what the implications of that are, especially in regard to rustls vs native-tls which could be problematic.
There was a problem hiding this comment.
It also adds a bunch of deps but I think most of them are inevitable.
| let client = Client::new(url)?; | ||
|
|
||
| let (org, project) = get_org_project_id(Api::current(), org, project)?; | ||
| let session = Usecase::new("preprod") |
There was a problem hiding this comment.
We should set an expiration policy here.
I think based on our discussions TTL should be ideal for this usecase?
There was a problem hiding this comment.
As we discussed we want different TTL based on customer/plan. We could introduce a new endpoint in sentry that tells you this info and set the TTL here based on that.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 3 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| // For other files, use {org}/{project}/{hash} | ||
| let hash = compute_sha256_hash(&contents); | ||
| format!("{org}/{project}/{hash}") | ||
| }; |
There was a problem hiding this comment.
JSON file keys missing project ID causes cross-project collisions
High Severity
JSON file keys use the format {org}/{snapshot_id}/{filename} which excludes the project ID, while non-JSON files use {org}/{project}/{hash} which includes it. Since the command requires --project and is documented as uploading "build snapshots to a project", omitting the project ID from JSON file keys causes different projects within the same organization to overwrite each other's files when using the same snapshot_id and filename.
| files.push(entry_path.to_path_buf()); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Hidden directories not skipped, exposing sensitive files
Medium Severity
The WalkDir traversal descends into hidden directories (e.g., .git, .svn, .env.d/) but is_hidden_file only checks if the file's own name starts with .. Files inside hidden directories with non-hidden names (like .git/config, .git/objects/*) are uploaded despite the intent to skip hidden content. This could inadvertently upload sensitive data or significantly bloat uploads with unwanted files.
Additional Locations (1)
| .file_name() | ||
| .and_then(|name| name.to_str()) | ||
| .unwrap_or("unknown.json"); | ||
| format!("{org}/{snapshot_id}/{filename}") |
There was a problem hiding this comment.
JSON files in subdirectories with same filename overwrite each other
Medium Severity
For JSON files, the key is generated using only file_name() which extracts just the base filename, discarding any subdirectory path. If a user uploads a directory containing subdir1/data.json and subdir2/data.json, both files produce the identical key {org}/{snapshot_id}/data.json, causing the second file to silently overwrite the first and resulting in data loss.
|
Hey @lcian, are you ready for me to review this, or are you still planning to check the feedback from Bugbot and/or iterate further? |
3 participants
Updated version of #3049 to discuss and iterate on things.
Notable changes:
shard_indexparameter from the command. I'm not sure what the purpose of that was originally.many(batch) API fromobjectstore_client. All uploads are executed as batch requests, reducing network overhead. Unfortunately, with they way things are implemented now, we will still have to buffer all files in memory before sending the request, as we need to hash their contents to determine the filename. If we could just use the filename as the key in objectstore, it would be much better because that way we could stream the files over.Note that auth enforcement still needs to be enabled for objectstore, so that's currently blocking this to be used for anything but internal testing.
Ref FS-233