Support raw RGB tiled TIFF by gigony · Pull Request #108 · rapidsai/cucim

gigony · 2021-09-28T22:37:50Z

Due to #17 ([BUG] Compression scheme 33003 tile decoding is not implemented), #19 (Check compression method used in the image) was merged so cuCIM has been handling only particular formats (only jpeg/deflate-compressed image).

If the input TIFF image has no-compressed RAW tile image, cuCIM showed the following error message:

RuntimeError: This format (compression: 1, sample_per_pixel: 1, planar_config: 1, photometric: 1) is not supported yet!. https://github.com/Project-MONAI/MONAI/pull/2987/checks?check_run_id=3667530814#step:7:15641

This patch is to support raw RGB tiled TIFF with fast-path, re-allowing non-compressed image with slow-path (in case the input is not tiled RGB image).

This supports Project-MONAI/MONAI#2987.

grlee77

This looks good, but is there a simple test we can add to ensure it is working as expected? Maybe read a simple raw TIFF generated in PIL via Python?

gigony · 2021-09-28T23:02:23Z

This looks good, but is there a simple test we can add to ensure it is working as expected? Maybe read a simple raw TIFF generated in PIL via Python?

Thanks @grlee77 for the review!
Sure. Will add a test case soon.

gigony · 2021-09-28T23:22:22Z

@grlee77 Added unit tests for this : 537c76e

grlee77

Thanks, it looks like the test failures here will be resolved by #110

jakirkham

Thanks Gigon! 😄

Had a couple questions above

jakirkham · 2021-09-29T18:23:19Z

+                unsigned char* raw_buf,
+                uint64_t offset,
+                uint64_t size,
+                uint8_t** dest,


Why does this need to be ** as opposed to *?

For the same reason that cudaMalloc accept '**' (https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY.html#group__CUDART__MEMORY_1g37d37965bfb4803b6d4e59ff26856356) -- decoders may need to allocate memory and setting the address to dest if needed.

See

cucim/cpp/plugins/cucim.kit.cuslide/src/cuslide/jpeg/libjpeg_turbo.cpp

Line 147 in 048d53e

if ((*dest = (unsigned char*)tjAlloc(width * height * tjPixelSize[pixelFormat])) == nullptr)

Was mostly thinking about this in terms of the nullptr checks below. We may want to look into using references for this kind of thing in the future, but not urgent. Can be addressed later

jakirkham · 2021-09-29T18:24:25Z

+    case COMPRESSION_NONE:
+    case COMPRESSION_JPEG:
+    case COMPRESSION_ADOBE_DEFLATE:
+    case COMPRESSION_DEFLATE:
+        return true;
+    default:
+        return false;


Might be worth indenting these to make this a bit more readable

~~Thanks! Will do.~~

It turns out that the style is auto-formatted by the current clang-formatter setting (.clang-format) so this is the right style as long as we update clang-format configuration.

Ah ok. Maybe we can track C++ formatting cleanup as an issue

Yeah, will create an issue for that.

created an issue for that: #116

gigony

Thanks @jakirkham for the review! Will update it soon.

gigony · 2021-09-29T21:00:55Z

+                unsigned char* raw_buf,
+                uint64_t offset,
+                uint64_t size,
+                uint8_t** dest,


For the same reason that cudaMalloc accept '**' (https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__MEMORY.html#group__CUDART__MEMORY_1g37d37965bfb4803b6d4e59ff26856356) -- decoders may need to allocate memory and setting the address to dest if needed.

See

cucim/cpp/plugins/cucim.kit.cuslide/src/cuslide/jpeg/libjpeg_turbo.cpp

Line 147 in 048d53e

if ((*dest = (unsigned char*)tjAlloc(width * height * tjPixelSize[pixelFormat])) == nullptr)

gigony · 2021-09-29T21:03:01Z

+    case COMPRESSION_NONE:
+    case COMPRESSION_JPEG:
+    case COMPRESSION_ADOBE_DEFLATE:
+    case COMPRESSION_DEFLATE:
+        return true;
+    default:
+        return false;


~~Thanks! Will do.~~

It turns out that the style is auto-formatted by the current clang-formatter setting (.clang-format) so this is the right style as long as we update clang-format configuration.

jakirkham · 2021-09-29T22:08:45Z

rerun tests

(looks like one of the jobs failed due to some sort of Jenkins issue)

jakirkham · 2021-09-30T00:33:03Z

@gpucibot merge

gigony added the feature request New feature or request label Sep 28, 2021

gigony added this to the v21.10 milestone Sep 28, 2021

gigony requested review from jakirkham and quasiben September 28, 2021 22:37

gigony requested review from a team as code owners September 28, 2021 22:37

gigony self-assigned this Sep 28, 2021

gigony added the non-breaking Introduces a non-breaking change label Sep 28, 2021

grlee77 reviewed Sep 28, 2021

View reviewed changes

grlee77 approved these changes Sep 29, 2021

View reviewed changes

gigony force-pushed the support_nocompression branch from 537c76e to 3049233 Compare September 29, 2021 18:09

gigony requested a review from a team as a code owner September 29, 2021 18:09

jakirkham reviewed Sep 29, 2021

View reviewed changes

gigony commented Sep 29, 2021

View reviewed changes

gigony mentioned this pull request Sep 29, 2021

[FEA] Add style checks in CI/CD for C++ code #116

Open

jakirkham approved these changes Sep 29, 2021

View reviewed changes

gigony and others added 3 commits September 29, 2021 16:07

Support raw RGB tiled TIFF

433bbc2

Add unit tests for RAW tiled TIFF image

9efa27b

Check nullptr for decoder's dest parameter

b9df835

gigony force-pushed the support_nocompression branch from 7e81eb4 to b9df835 Compare September 29, 2021 23:07

rapids-bot Bot merged commit 4d861fe into rapidsai:branch-21.10 Sep 30, 2021

Conversation

gigony commented Sep 28, 2021

Uh oh!

grlee77 left a comment

Choose a reason for hiding this comment

Uh oh!

gigony commented Sep 28, 2021

Uh oh!

gigony commented Sep 28, 2021

Uh oh!

grlee77 left a comment

Choose a reason for hiding this comment

Uh oh!

jakirkham left a comment

Choose a reason for hiding this comment

Uh oh!

jakirkham Sep 29, 2021

Choose a reason for hiding this comment

Uh oh!

gigony Sep 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakirkham Sep 29, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jakirkham Sep 29, 2021

Choose a reason for hiding this comment

Uh oh!

gigony Sep 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakirkham Sep 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gigony Sep 29, 2021

Choose a reason for hiding this comment

Uh oh!

gigony Sep 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gigony left a comment

Choose a reason for hiding this comment

Uh oh!

gigony Sep 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gigony Sep 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakirkham commented Sep 29, 2021

Uh oh!

jakirkham commented Sep 30, 2021

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gigony Sep 29, 2021 •

edited

Loading

gigony Sep 29, 2021 •

edited

Loading

jakirkham Sep 29, 2021 •

edited

Loading

gigony Sep 29, 2021 •

edited

Loading

gigony Sep 29, 2021 •

edited

Loading

gigony Sep 29, 2021 •

edited

Loading