Skip to content

feat(web): remove upload file limit with rxjs and improve import size - #1743

Merged
alextran1502 merged 5 commits into
mainfrom
feat/web-remove-upload-file-limit-with-rsjx
Feb 13, 2023
Merged

alextran1502 merged 5 commits into
mainfrom
feat/web-remove-upload-file-limit-with-rsjx

Conversation

@alextran1502

Copy link
Copy Markdown
Member

No description provided.

@vercel

vercel Bot commented Feb 13, 2023 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated
immich-code-coverage ✅ Ready (Inspect) Visit Preview 💬 Add your feedback Feb 13, 2023 at 5:15PM (UTC)
1 Ignored Deployment
Name Status Preview Comments Updated
immich ⬜️ Ignored (Inspect) Feb 13, 2023 at 5:15PM (UTC)

@alextran1502
alextran1502 requested a review from jrasm91 February 13, 2023 06:24
@vercel
vercel Bot temporarily deployed to Preview – immich-code-coverage February 13, 2023 06:25 Inactive
@vercel
vercel Bot temporarily deployed to Preview – immich-code-coverage February 13, 2023 06:27 Inactive
@michelheusschen

michelheusschen commented Feb 13, 2023 •

Copy link
Copy Markdown
Collaborator

The upload notifications don't play along nicely. When uploading some smaller files, the upload notification only appears for a split second.

For limiting the concurrency of uploading files, I would like to propose another solution without having to import rxjs:

export const fileUploadHandler = async (
	files: File[],
	albumId: string | undefined = undefined,
	sharedKey: string | undefined = undefined,
	onDone?: (id: string) => void
) => {
	const concurrency = 2;
	const acceptedFiles = files
		.filter((file) => {
			const assetType = getFileMimeType(file).split('/')[0];
			return assetType === 'video' || assetType === 'image';
		})
		.entries();

	// Start new workers up to the set concurrency limit. Each worker will take
	// files from the shared acceptedFiles iterator.
	const workers = Array<typeof acceptedFiles>(concurrency)
		.fill(acceptedFiles)
		.map(async (files) => {
			for (const [, file] of files) {
				await fileUploader(file, albumId, sharedKey, onDone);
			}
		});

	// Wait until all workers have completed.
	await Promise.all(workers);
};

@alextran1502

alextran1502 commented Feb 13, 2023 •

Copy link
Copy Markdown
Member Author

The upload notifications don't play along nicely. When uploading some smaller files, the upload notification only appears for a split second.

For limiting the concurrency of uploading files, I would like to propose another solution without having to import rxjs:

export const fileUploadHandler = async (
	files: File[],
	albumId: string | undefined = undefined,
	sharedKey: string | undefined = undefined,
	onDone?: (id: string) => void
) => {
	const concurrency = 2;
	const acceptedFiles = files
		.filter((file) => {
			const assetType = getFileMimeType(file).split('/')[0];
			return assetType === 'video' || assetType === 'image';
		})
		.entries();

	// Start new workers up to the set concurrency limit. Each worker will take
	// files from the shared acceptedFiles iterator.
	const workers = Array<typeof acceptedFiles>(concurrency)
		.fill(acceptedFiles)
		.map(async (files) => {
			for (const [, file] of files) {
				await fileUploader(file, albumId, sharedKey, onDone);
			}
		});

	// Wait until all workers have completed.
	await Promise.all(workers);
};

@michelheusschen

I was experimenting with removing the setTimeout to remove the finished item from the UI list, so that must be why it flashes for small files.

Thank you for the suggestion, I will try it in the code.

FYI, I try to use the openapi client to perform the upload action, but the payload somehow indicate the AssetType as binary even though we pass in an enum. So the server doesn't accept it. Might worth looking into in a separate PR.

@vercel
vercel Bot temporarily deployed to Preview – immich-code-coverage February 13, 2023 17:06 Inactive
@vercel
vercel Bot temporarily deployed to Preview – immich-code-coverage February 13, 2023 17:15 Inactive
@alextran1502 alextran1502 changed the title feat(web): remove upload file limit with rxjs feat(web): remove upload file limit with rxjs and improve import size Feb 13, 2023
@alextran1502

Copy link
Copy Markdown
Member Author

@michelheusschen FYI, I discussed with the group, and we are going with RxJs route as it is easier to understand, and with tree shaking, we are not loading a lot of code from the library itself.

@alextran1502
alextran1502 merged commit 2c1aab1 into main Feb 13, 2023
@alextran1502
alextran1502 deleted the feat/web-remove-upload-file-limit-with-rsjx branch February 13, 2023 19:18

for (const asset of acceptedFile) {
await fileUploader(asset, albumId, sharedKey, onDone);
for (const file of acceptedFile) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lines above this also loop through the list of assets. Can we combine the two loops into one? ie for each asset, get mime type, if true add to files$?

This branch was previously deployed

1 inactive deployment
Preview – immich-code-coverage — d877a421 Deployed Feb 13, 2023 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants