SMARTS(二).1 straight 环境 + agent (PPO)训练代码解读--环境配置

SMARTS启动straight 环境 + agent (PPO)训练

将smarts/scenarios/straight文件夹copy到smarts/benchmark/scenarios/下面，并将copy过后的smarts/benchmark/scenarios/straight/scenario.py替换为下面的内容：

from smarts.sstudio import types as t
from smarts.sstudio import gen_scenario

trajectory_boid_agent = t.BoidAgentActor(
    name="trajectory-boid",
    agent_locator="scenarios.straight.agent_prefabs:trajectory-boid-agent-v0",
)

pose_boid_agent = t.BoidAgentActor(
    name="pose-boid",
    agent_locator="scenarios.straight.agent_prefabs:pose-boid-agent-v0",
)

traffic = t.Traffic(
    flows=[
        t.Flow(
            route=t.Route(begin=("west", lane_idx, 30), end=("east", lane_idx, "max"),),
            rate=50,
            actors={
   
   t.TrafficActor("car"): 1},
        )
        for lane_idx in range(3)
    ]
)
missions = [
    t.Mission(t.Route(begin=("west", 1, 10), end=("east", 1, 90))),
]

scenario = t.Scenario(
    traffic={
   
   "all": traffic},
    ego_missions=missions,
    bubbles=[
        t.Bubble(
            zone=t.PositionalZone(pos=(50, 0), size=(40, 20)),
            margin=5,
            actor=trajectory_boid_agent,
        ),
        t.Bubble(
            zone=t.PositionalZone(pos=(150, 0), size=(50, 20)),
            margin=5,
            actor=pose_boid_agent,
            keep_alive=True,
        ),
    ],
)

gen_scenario(scenario, output_dir=str(Path(__file__).parent))

运行下列语句，启动训练过程&在浏览器中打开http://localhost:8081/#/便可以可视化查看训练过程。

# terminal 1 
scl envision start -s ./scenarios -p 8081
# terminal 2
python run.py benchmark/scenarios/intersections/straight -f agents/ppo/baseline-continuous-control.yaml

在这里实例中为例说明scenario文件的定义。
接口函数为run.py，在benchmark的目录下。

straight环境定义

该环境下具有3个社会车辆(traffic提供路上的基础交通流），一个待训练车辆(ego vehicle)。

各种环境的定义都在sstudio.py文件中，其中types.py文件即为各种类型的定义，包括Flow， bubble，traffic，Scenario，Route和TrafficActor等等，请注意：这个types文件夹中只是实现了类型。尽管各个类型均采用class方式实现，但是终究只是与int，char等是同样地位和功能的变量而已。 具体的解析和操作需要查看其他操作这个类型的脚本文件。

该文件应该从后往前阅读，gen_scenario用于设置好的环境scenario，并将对应的日志文件输出到指定的文件夹中。

环境定义

整个scenario.py就是定义了一个Scenario类并用gen_scenario()解析&运行这个Scenario类。Scenario的类定义如下：

@dataclass(frozen=True)
class Scenario:
    traffic: Optional[Dict[str, Traffic]] = None
    ego_missions: Optional[Sequence[Mission]] = None
    # e.g. { "turning_agents": ([actors], [missions]), ... }
    social_agent_missions: Optional[
        Dict[str, Tuple[Sequence[SocialAgentActor], Sequence[Mission]]]
    ] = None
    """Every dictionary item {group: (actors, missions)} gets run simultaneously. If
    actors > 1 and missions = 0 or actors = 1 and missions > 0 we cycle through them
    every episode. Otherwise actors must be the same length as missions.
    """
    bubbles: Optional[Sequence[Bubble]] = None
    friction_maps: Optional[Sequence[RoadSurfacePatch]] = None
    traffic_histories: Optional[Sequence[str]] = None

接下来具体分析Scenario的每一个成员。

Traffic

Scenario类型定义方式

@dataclass(frozen=True)
class Scenario:
    traffic: Optional[Dict[str, Traffic]] = None

其中dataclass代表定义的类为数据类，其中frozen代表初始化之后属性数值便不可再改；

traffic使用例子：

from smarts.sstudio import types as t
from smarts.sstudio import gen_scenario
traffic = t.Traffic(
    flows=[
        t.Flow(
            route=t.Route(begin=("west", lane_idx, 30), end=("east", lane_idx, "max"),),
            rate=50,
            actors={
   
   t.TrafficActor("car"): 1},
        )
        for lane_idx in range(3)
    ]
)
scenario = t.Scenario(
    traffic={
   
   "all": traffic},
)
gen_scenario(scenario, output_dir=str(Path(__file__).parent))

Traffic类本身定义如下，只有flows一个成员。

@dataclass(frozen=True)
class Traffic:
    """The descriptor for traffic."""

    flows: Sequence[Flow]
    """Flows are used to define a steady supply of vehicles."""

其中Sequence是list和tuple的泛化类型，即没有必要区分tuple和list所以就定义一个类型把他们都涵盖，这个涵盖list和tuple的类型即Sequence

Traffic类成员1：Flow定义内容为

@dataclass(frozen=True)
class Flow:
    """A route with an actor type emitted at a given rate."""

    route: Route
    """The route for the actor to attempt to follow."""
    rate: float
    """Vehicles per hour."""
    begin: float = 0
    """Start time in seconds."""
    # XXX: Defaults to 1 hour of traffic. We may want to change this to be "continual
    #      traffic", effectively an infinite end.
    end: float = 1 * 60 * 60
    """End time in seconds."""
    actors: Dict[TrafficActor, float] = field(default_factory=dict)
    """An actor to weight mapping associated as\\: { actor: weight }
    actor:
        The traffic actors that are provided.
    weight:
        The chance of this actor appearing as a ratio over total weight.
    """

Flow成员1： Route的数据结构定义如下

@dataclass(frozen=True)
class Route:
    """A route is represented by begin and end edge IDs, with an optional list of
    itermediary edge IDs. When an intermediary is not specified the router will
    decide what it should be.
    """

    ## edge, lane index, offset
    begin: Tuple[str, int, Any