Skip to content

[Bug] adding ColorLog override will disable other overrides #1706

Closed

Description

@NinjaAtWork

Hi
here i follow the basic tutorial and experiment section :
main.py:

from omegaconf import DictConfig, OmegaConf
import hydra

@hydra.main(config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg))

if __name__ == "__main__":
    my_app()

config.yaml:

defaults:
  - segment : color
  #- override hydra/job_logging: colorlog
  #- override hydra/hydra_logging: colorlog
volume: 1

segment/color.yaml :
color_num=1
segment/color2.yaml :
color_num=10

experiment/test.yaml :

# @package _global_
defaults:
  - override /segment: color2
volume: 9999

running the
:\python main.py +experiment=test
will return volume:9999
BUT while un-commenting colorlog override in config.yaml the volume returned will be volume:1
why adding colorlog impact the other config ?
btw the colorlog do his job and the colorizing the output

thanks

Activity

  1. jieru-hu commented on Jul 8, 2021

    @jieru-hu
    Contributor

    Hi @NinjaAtWork

    I created this repro based on the info provided.

    This does look like a bug - I could reproduce your example locally. If we run --info defaults-tree on the example app, we can see that the defaults tree are generated differently in two cases.

    without override hydra/job_logging in primary config

    Defaults Tree
    *************
    <root>:
      hydra/config:
        hydra/output: default
        hydra/launcher: basic
        hydra/sweeper: basic
        hydra/help: default
        hydra/hydra_help: default
        hydra/hydra_logging: default
        hydra/job_logging: default
        hydra/callbacks: null
        hydra/env: default
        _self_
      config:
        segment: color2
        _self_
        experiment: test
    

    with override hydra/job_logging in primary config

    Defaults Tree
    *************
    <root>:
      hydra/config:
        hydra/output: default
        hydra/launcher: basic
        hydra/sweeper: basic
        hydra/help: default
        hydra/hydra_help: default
        hydra/hydra_logging: colorlog
        hydra/job_logging: colorlog
        hydra/callbacks: null
        hydra/env: default
        _self_
      config:
        segment: color2
        experiment: test
        _self_
    

    For now, to get around this, you can try adding the _self_ keyword to your config.yaml file. link to doc

    defaults:
      - _self_
      - /segment: color
      - override /hydra/job_logging: colorlog
      - override /hydra/hydra_logging: colorlog
    

    Please give this a try and let us know if it helps. We will also take a look at this issue and apply fix if needed.

  2. added this to the Hydra 1.1.1 milestone on Jul 8, 2021
  3. NinjaAtWork commented on Jul 9, 2021

    @NinjaAtWork
    Author

    Hi @jieru-hu

    adding _self_ to start of the defaults worked and thanks for the solution and your time for this. 💯
    just for fun: i read all the docs , as someone who like architecture more than programing , my brain will prefer skip when processing words which start with underline like _ self _ or __ main __ or same others ,you know 😄
    please close this issue after a few days which i will be able to report if something somewhere goes tricky or need help.
    kindly , by far ,when i use colorlog , then i am unable to use custom job_logging which is described in docs. its great to be able to switch between save to files and save to prompt in colorlog mode .let me know its solution too .

  4. jieru-hu commented on Jul 9, 2021

    @jieru-hu
    Contributor

    Hi @NinjaAtWork

    Do you mean you want to have both colorlog and somehow also follow this example?

    If so, you can certainly create your own logging configuration. You can extend the colorlog's config and override the any field as needed (all the Hydra loggings configurations are passed in to logging.config)

    Here is the documentation on how to extend a plugin config https://hydra.cc/docs/patterns/configuring_plugins#extending-plugin-default-config

  5. Jasha10 commented on Jul 16, 2021

    @Jasha10
    Contributor

    This does look like a bug - I could reproduce your example locally. If we run --info defaults-tree on the example app, we can see that the defaults tree are generated differently in two cases.

    without override hydra/job_logging in primary config

    ...

    with override hydra/job_logging in primary config

    ...

    @jieru-hu The desired behavior is that both cases should look like the first defaults tree, right?

    I'm going to start working on a fix for this soon (unless you've already got work in progress :) ).

  6. omry commented on Jul 17, 2021

    @omry
    Collaborator

    Yes, I think this is what I would expect in this situation for the user's config section:

      config:
        segment: color2
        _self_
        experiment: test
    
  7. NinjaAtWork commented on Jul 17, 2021

    @NinjaAtWork
    Author

    @jieru-hu

    my custom job logging as you mentioned but this will disable the color effect :

    defaults:
      - colorlog
    version: 1
    formatters:
      simple:
        format: '[%(levelname)s] - %(message)s'
    handlers:
      console:
        class: logging.StreamHandler
        formatter: simple
        stream: ext://sys.stdout
    root:
      handlers: [console]
    
    disable_existing_loggers: false
    

    i think this will be linked with that minor bug of override. ill wait for new update and try this again. :)

  8. jieru-hu commented on Jul 19, 2021

    @jieru-hu
    Contributor

    @NinjaAtWork I'm not sure if this is related to this particular bug. You overrides actually disable the colorlog, in particular this line

      console:
        class: logging.StreamHandler
        formatter: simple
        stream: ext://sys.stdout
    

    I've verified this by copy pasting your config and update formatter: colorlog.

  9. self-assigned this
    on Jul 19, 2021
  10. NinjaAtWork commented on Jul 20, 2021

    @NinjaAtWork
    Author

    @jieru-hu
    replacing formatter from simple to colorlog will color the console but also change format from '[%(levelname)s] - %(message)s' to colorlog format which include date and time.
    i think when user able to define colorlog as defaults in first line then could change format to something desired
    all i need is to do something like this :

    formatters:
      colorlog:
        format: '[%(levelname)s] - %(message)s'
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

In progressWork in progressbugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions