[WIP] Load simulator memory from file

[baltazarortiz]

Related issue: n/a

Type of change: new feature

Impact: software change

Release Notes

WIP: Still needs debugging, but discussing this and #591 in the same mailing list thread was getting confusing. I'm guessing my c++ isn't great but I'm happy to clean it up after the functionality is correct!

In this branch + the branch of testchipip that is currently being pointed to for testing purposes, I have added command line support to initialize the simulated DRAM.

Usage (also added to the simulator help output): ./simulator-chipyard-RocketConfig +dramsim +fastloadmem --loadmem=/path/to/test.hex /path/to/test.riscv

I attempted to do this as similarly as possible to the way that Firesim does (see here and here). Currently, the load finishes, but then the simulator segfaults during the first call to memory_tick().

I have tried passing a file from generate_memory_init.py as well as a file that only has a single line, and I get the same result. I also get the same result whether or not I pass the +dramsim flag (though it does show the dramsim ini files being loaded if I do).

[davidbiancolin]

Cool! You can have a look at @colinschmidt's pr #598 to see how to more cleanly add new plusargs.

[baltazarortiz]

Thanks! I'll check that out. Any ideas about the segfault? Is there a different format of input file I should be using?

[baltazarortiz]

I switched the plusarg parsing to match #598, thanks for pointing this out!

I'm still unsure about why the model segfaults as soon as the load finishes. I remembered that I had to replace MEM_DATA_BITS with a constant - is there any equivalent in Chipyard to the automatic header generation in midas? I didn't see anything obvious.

[baltazarortiz]

I thought I had a hack that worked for my system (blackboxed MMIO peripheral containing a huge memory I could directly initialize in the verilog), but it's looking to be insufficient. @davidbiancolin any ideas on things I should look into to debug this approach crashing / any examples of what using this flag on firesim looks like? I still haven't been able to find any examples.

[zhemao]

This line here needs one more level of indirection.

load_mem((void**)mm->get_data(), loadmem.c_str(), 64 / 8, 1);

The load_mem function expects an array of pointers, one to each memory channel. If you don't care about multichannel, you can change it to this:

load_mem((void **) &mm->get_data(), loadmem.c_str(), 64, 1);

Also, note that the line_size argument is specified in bytes and not in words.

[zhemao]

The proper way of doing this that makes it compatible with multichannel would be a bit trickier. We'd have to rearchitect SimDRAM so that it knows about all the channels.

[baltazarortiz]

Thanks! I'll check this out. I personally do not need multichannel, just some way to load memory. It definitely looks like making this PR fully compatible in general would require a lot of changes to automatically pull parameters out of Chipyard, since the ones the firesim implementation grabs things from headers that come out of Midas/golden gate if I understand correctly?

[baltazarortiz]

To confirm, is the format of file that comes out of generate_memory_init.py the correct format to load via this function?

[zhemao]

Yes, it should work. The format is just a list of hex numbers in little-endian order.

[baltazarortiz]

I updated the argument to be a double pointer and changed line_size to be in bytes. It isn't segfaulting now but also doesn't seem to be loading anything.

I tried providing it a file that is entirely filled with f's and then running a hello world + memory print:

#include <stdio.h>                                                              
#include <stdint.h>                                                             
int main() {                                                                    
        printf("Hello, World!\n");                                                   
              
        #define RAM ( *(volatile uint32_t*)0x80000000 )                                                 
        printf("RAM val: %x\n", RAM);                                           
                     
        return 0;                                                               
} 

However, the print is consistently RAM val: 41014081. My SoC is set up with a default memory starting at 0x80000000, but I'm linking my application to start at 0x85000000. I'll make a separate TLRAM for loading my program into to see if something about the way FESVR loads applications that is overwriting the initialized memory.

[baltazarortiz]

I wasn't able to get the TLRAM to store the program I'm trying to run, but I did find that if I change my link address to 0x80000000, the program still runs fine, which presumably means no memory is being loaded (I would assume that the program would be overwritten and not be able to run?)

[zhemao]

HTIF generally runs after loadmem, so it's most likely overwriting the memory with the program.

Are you invoking it with the correct arguments? In the invocation you provided, you use --loadmem=, but your code is expecting +loadmem=.

[baltazarortiz]

That makes sense, does HTIF start writing from the beginning of memory though? I would expect that to mean that I'd see significantly slower load times for loading at 0x85000000 vs 0x80000000, but (without timing exactly) that doesn't seem to be the case.

I am, good catch - --loadmem= was my initial implementation before I updated based on #598. I'm invoking with

make run-binary-debug CONFIG=MyRocketConfig BINARY=../../tests/hello.riscv SIM_FLAGS="+fastloadmem +loadmem=/home/ubuntu/test.hex"

which results in the simulator invocation being

/home/ubuntu/chipyard/sims/verilator/simulator-chipyard-MyRocketConfig-debug +permissive +fastloadmem +loadmem=/home/ubuntu/test.hex +dramsim +dramsim_ini_dir=/home/ubuntu/chipyard/generators/testchipip/src/main/resources/dramsim2_ini +max-cycles=10000000  +verbose -v/home/ubuntu/chipyard/sims/verilator/output/chipyard.TestHarness.MyRocketConfig/hello.vcd +permissive-off ../../tests/hello.riscv

The print statement I added indicating that the fastloadmem is happening is also popping up, so the code is definitely being reached. Just unsure if the load is happening correctly.

[colinschmidt]

Closing this as the dev branch now includes #635 and #639 which should allow you to use loadmem and also have HTIF skip memory that has been loaded via the sideband.