Skip to content

Implement RSSReport class - #7349

Merged
dsouzai merged 1 commit into
eclipse-omr:masterfrom
gita-omr:rss_report
Jul 10, 2024
Merged

dsouzai merged 1 commit into
eclipse-omr:masterfrom
gita-omr:rss_report

Conversation

@gita-omr

Copy link
Copy Markdown
Contributor
  • Implement a class that can monitor RSS status of any number of arbitry memory regions
  • In addition to start address and size, region can have name, as well as a list of items describing its content

@gita-omr

Copy link
Copy Markdown
Contributor Author

Fixed white spaces.

@gita-omr

Copy link
Copy Markdown
Contributor Author

Removed Options.hpp include since it's not needed.

@gita-omr

Copy link
Copy Markdown
Contributor Author

Build fixes.

@gita-omr

Copy link
Copy Markdown
Contributor Author

Fix for windows build.

@gita-omr

Copy link
Copy Markdown
Contributor Author

Another Windows build fix.

@gita-omr gita-omr changed the title Implement and use RSSReport class Implement RSSReport class May 27, 2024
@gita-omr

Copy link
Copy Markdown
Contributor Author

More Windows build fixes.

@gita-omr

Copy link
Copy Markdown
Contributor Author

Another Windows warning.

@gita-omr

Copy link
Copy Markdown
Contributor Author

another warning

@gita-omr

Copy link
Copy Markdown
Contributor Author

Corrected second commit

@gita-omr

Copy link
Copy Markdown
Contributor Author

I will fix line endings together with PR comments.

@gita-omr

Copy link
Copy Markdown
Contributor Author

Fixed Mac build and line endings.

@gita-omr

Copy link
Copy Markdown
Contributor Author

FYI: @mpirvu @tajila

Note: further improvements are possible:

  1. options for reporting only specific types of memory (e.g. data cache only, code cache only etc.)
  2. disclaiming regions from within the class

@gita-omr

Copy link
Copy Markdown
Contributor Author

I think I addressed all the comments and I am not planning on adding any new interfaces to that class right now. @dsouzai @0xdaryl could you please take another look?

@dsouzai dsouzai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gita-omr

Copy link
Copy Markdown
Contributor Author

I still need to write a brief document for it.

@gita-omr

gita-omr commented Jul 3, 2024

Copy link
Copy Markdown
Contributor Author

Improved printing format and added a document. @0xdaryl @dsouzai please kindly take a look so that we can finally merge it. All further enhancements will be in a different PR.

@gita-omr

gita-omr commented Jul 3, 2024

Copy link
Copy Markdown
Contributor Author

Small correction in the doc.

@dsouzai

dsouzai commented Jul 8, 2024

Copy link
Copy Markdown
Contributor

Doc looks ok to me; after Daryl's taken a look, could you squash the extra "fix" commits?

}

#ifdef LINUX
typedef struct __attribute__ ((__packed__))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you confirm this attribute will work on all Linux build compilers supported by OMR (https://github.com/eclipse/omr/blob/master/doc/SupportedC%2B%2BFeatures.md#minimum-compiler-support)?

I'm pretty sure packed is supported for gcc and Clang, but I'm less sure about the commercial xlC (not the one based on Clang) that we list as a supported compiler for OMR. It might work, but please confirm.

If not, you'll have to add some GNU and Clang ifdefs around it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The document does not mention packed. Can we assume all compilers support it? Otherwise, I am not sure how to check all of them. @0xdaryl

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably, that attribute is required for efficiency rather than correctness. If so, then I think guarding the typedef using #ifdef compiler macros just like in the example here should suffice:
https://github.com/eclipse/omr/blob/d58019eba46edbd5f2e1963e687a27c1dbaccac0/compiler/infra/String.hpp#L31
The #else part could just be a typedef without the attribute. However, if the attribute is required for correctness for some reason then put an #error in the #else part instead.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The attribute is needed for correctness. If some compiler does not support it I think it will give an error. Also, I noticed this attribute is being used in openj9 without any compiler check.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I don't think there are any non-Clang or non-gcc based compilers building and testing OMR. The reference to xlC 12 in the supported compilers list is a bit dated, and we actually plan on revisiting that list at the next OMR architecture meeting. Newer xlC's are based on Clang, I believe. So I'll let this go.

Comment thread compiler/runtime/OMRRSSReport.cpp Outdated
{
if ((regionNum * REGION_PRINT_WIDTH + 1) < MAX_RSS_LINE)
{
sprintf(rssLine + regionNum * REGION_PRINT_WIDTH, "%*.*s", REGION_PRINT_WIDTH, REGION_PRINT_WIDTH, rssRegion->_name);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sprintf has been causing grief with newer build compilers due to potential overflow issues being flagged as errors. I think it's best not to introduce new uses and just go with snprintf instead.

See #6863 and a comment from Devin on some ways of consolidating snprintf behaviour across compilers: #6863 (comment)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced sprintf with TR::snprintfTrunc.

Comment thread compiler/runtime/OMRRSSReport.cpp Outdated
{
if ((regionNum * REGION_PRINT_WIDTH + 1) < MAX_RSS_LINE)
{
sprintf(rssLine + regionNum * REGION_PRINT_WIDTH, "%*p", REGION_PRINT_WIDTH, rssRegion->_start);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snprintf, see earlier comment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced sprintf with TR::snprintfTrunc.

Comment thread compiler/runtime/OMRRSSReport.cpp Outdated

if ((regionNum * REGION_PRINT_WIDTH + 1) < MAX_RSS_LINE)
{
sprintf(rssLine + regionNum * REGION_PRINT_WIDTH, " %5zu/%5zu(%3.0f%%)", rssSizeKb, sizeKb, (float)rssSizeKb*100.0/sizeKb);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snprintf, see earlier comment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced sprintf with TR::snprintfTrunc.

Comment thread compiler/runtime/OMRRSSReport.cpp Outdated
if ((pageCount * PAGE_PRINT_WIDTH + PAGE_PRINT_WIDTH + 1) < MAX_RSS_LINE)
{
// Note: TR_VerboseLog will truncate the line to some maximum
sprintf(pagesInfo + pageCount * PAGE_PRINT_WIDTH, "%1zu ", resident);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snprintf, see earlier comment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced sprintf with TR::snprintfTrunc.

@gita-omr

gita-omr commented Jul 9, 2024 •

Copy link
Copy Markdown
Contributor Author

Replaced sprintf with TR::snprintfTrunc. Since we are printing RSS info in a tabular format (the title and then rows by rows) this provided an additional benefit of ensuring each field does not exceed its intended length. Since we place each field in a fixed position and ensure it's not too wide the results will look as expected.

@gita-omr

gita-omr commented Jul 9, 2024

Copy link
Copy Markdown
Contributor Author

Rebased

@gita-omr

gita-omr commented Jul 9, 2024

Copy link
Copy Markdown
Contributor Author

Corrected and improved latest commit.

@gita-omr

gita-omr commented Jul 9, 2024 •

Copy link
Copy Markdown
Contributor Author

Doc looks ok to me; after Daryl's taken a look, could you squash the extra "fix" commits?

Sure. I have not squashed yet since several new features are waiting for this one and it's easier to rebase those branches when this branch is not squashed.

@0xdaryl 0xdaryl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the sprintfs.

- Implement a class that can monitor RSS status of any number
  of arbitry memory regions
- In addition to start address and size, region can have name,
  as well as a list of items describing its content
@gita-omr

Copy link
Copy Markdown
Contributor Author

Squashed all the commits.

@dsouzai

dsouzai commented Jul 10, 2024

Copy link
Copy Markdown
Contributor

jenkins build all

@dsouzai
dsouzai merged commit de311dd into eclipse-omr:master Jul 10, 2024
${CMAKE_CURRENT_LIST_DIR}/OMRCodeCacheManager.cpp
${CMAKE_CURRENT_LIST_DIR}/OMRCodeCacheMemorySegment.cpp
${CMAKE_CURRENT_LIST_DIR}/OMRCodeCacheConfig.cpp
${CMAKE_CURRENT_LIST_DIR}/OMRRSSReport.cpp

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't an equivalent change be made in makefiles like jitbuilder/build/files/common.mk for example?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is the reason internal builds are failing (e.g. z/OS):

 IEW2456E 9207 SYMBOL addRSSItem__Q2_3OMR9RSSRegionFPQ2_3OMR7RSSItemiPCc        
          UNRESOLVED.  MEMBER COULD NOT BE INCLUDED FROM THE DESIGNATED CALL    
          LIBRARY.                                                              
 IEW2456E 9207 SYMBOL _instance__Q2_3OMR9RSSReport UNRESOLVED.  MEMBER COULD NOT
          BE INCLUDED FROM THE DESIGNATED CALL LIBRARY. NAME SPACE = 3          
FSUM3065 The LINKEDIT step ended with return code 8.
/j9tmp/bld_74387/bld_zos_390/compiler/../compiler/build/rules/zos-xlc/common.mk:47: recipe for target '/j9tmp/bld_74387/bld_zos_390/compiler/../libj9jit29.so' failed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dsouzai added a commit to dsouzai/openj9 that referenced this pull request Jul 11, 2024
eclipse-omr/omr#7349 added a new file called
OMRRSSRegion.cpp. Although this file was added to the CMakeLists.txt for
OMR, there does not exist a common UMA config to add this file. As such,
the OpenJ9 build when using UMA is broken.

This commit fixes this by adding this file to the UMA config.

Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants