Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dromajo bootrom issue #917

Closed
kabylkas opened this issue Jul 11, 2021 · 9 comments
Closed

dromajo bootrom issue #917

kabylkas opened this issue Jul 11, 2021 · 9 comments

Comments

@kabylkas
Copy link

Hi!

It seems bootrom directory has been removed from the latest chipyard repo. However, dromajo is still trying to get the bootrom from that path.

chipyard/tools/dromajo/dromajo.mk

Line 10 in b5d0131

DROMAJO_ROM = $(base_dir)/bootrom/bootrom.rv64.img

Should this be changed to a DROMAJO_ROM = $(build_dir)/bootrom.rv64.img?

Thanks!
@kabylkas
Copy link
Author

kabylkas commented Jul 11, 2021 •
edited
Loading

For the purposes of forward progress, I speculated that DROMAJO_ROM should point to $(build_dir)/bootrom.rv64.img (VCS was segfaulting if I didn't do that). After changing the path, there is a co-sim fail thought :( The co-simulation fails when executing bootrom code (see below). Before digging into the issue, could you please confirm I am doing everything right 😀 Thanks!

testing $random 18faeb31 seed 3990867110
== Loading device model file '/chipyard/generators/testchipip/src/main/resources/dramsim2_ini/DDR3_micron_64M_8B_x4_sg15.ini' ==
== Loading system model file '/chipyard/generators/testchipip/src/main/resources/dramsim2_ini/system.ini' ==
===== MemorySystem 0 =====
CH. 0 TOTAL_STORAGE : 4096MB | 1 Ranks | 16 Devices per rank
DRAMSim2 Clock Frequency =666666666Hz, CPU Clock Frequency=100000000Hz
CORE0: 3 0x0000000000010040 (0x00000517) x10 0x0000000000010040 auipc   a0, 0x0
CORE0: 3 0x0000000000010044 (0xfc050513) x10 0x0000000000010000 addi    a0, a0, -64
csr_read: hartid=0 csr=0x305 val=0x0
csr_write: hardid=0 csr=0x305 val=0x0000000000010000
CORE0: 3 0x0000000000010048 (0x30551073)                        csrw    mtvec, a0
csr_read: hartid=0 csr=0x301 val=0x94112d
CORE0: 3 0x000000000001004c (0x301022f3) x5  0x800000000094112d csrr    t0, misa
CORE0: 3 0x0000000000010050 (0x4122d293) x5  0xffffe00000000025 srai    t0, t0, 18
CORE0: 3 0x0000000000010054 (0x0012f293) x5  0x0000000000000001 andi    t0, t0, 1
CORE0: 3 0x0000000000010058 (0x00028463)                        beqz    t0, pc + 8
csr_read: hartid=0 csr=0x303 val=0x0
csr_write: hardid=0 csr=0x303 val=0x0000000000000000
CORE0: 3 0x000000000001005c (0x30301073)                        csrw    mideleg, zero
CORE0: 3 0x0000000000010060 (0x00800513) x10 0x0000000000000008 li      a0, 8
csr_read: hartid=0 csr=0x304 val=0x0
csr_write: hardid=0 csr=0x304 val=0x0000000000000008
CORE0: 3 0x0000000000010064 (0x30451073)                        csrw    mie, a0
csr_read: hartid=0 csr=0x300 val=0x1800
csr_write: hardid=0 csr=0x300 val=0x0000000a00001808
CORE0: 3 0x0000000000010068 (0x30052073)                        csrs    mstatus, a0
CORE0: 3 0x000000000001006c (0x10500073)                        wfi
[DEBUG] DUT raised interrupt 3
[DEBUG] Interrupt: MIP <- 3: Now MIP = 8
get_irq_mask: mip=0x8 mie=0x8 mideleg=0x0
raise_interrupt: irq=3 priv=3 pc=10070 hartid=0
hartid=0: exception interrupt #3, epc 0x0000000000010070
CORE0: 3 0x0000000000010000 (0x020005b7) x11 0x0000000002000000 lui     a1, 0x2000
csr_read: hartid=0 csr=0xf14 val=0x0
CORE0: 3 0x0000000000010004 (0xf1402573) x10 0x0000000000000000 csrr    a0, mhartid
CORE0: 3 0x0000000000010008 (0x00050463)                        beqz    a0, pc + 8
CORE0: 3 0x0000000000010010 (0x00458613) x12 0x0000000002000004 addi    a2, a1, 4
CORE0: 3 0x0000000000010014 (0x00100693) x13 0x0000000000000001 li      a3, 1
clint_write: MSIP access for hartid:1 which is beyond ncpus
CORE0: 3 0x0000000000010018 (0x00d62023)                        sw      a3, 0(a2)
CORE0: 3 0x000000000001001c (0x00460613) x12 0x0000000002000008 addi    a2, a2, 4
clint_read: MSIP access for hartid:1 which is beyond ncpus
CORE0: 3 0x0000000000010020 (0xffc62683) x13 0x0000000000000000 lw      a3, -4(a2)
CORE0: 3 0x0000000000010024 (0xfe069ae3)                        bnez    a3, pc - 12
CORE0: 3 0x0000000000010028 (0x05c0006f)                        j       pc + 0x5c
CORE0: 3 0x0000000000010084 (0x0005a023)                        sw      zero, 0(a1)
CORE0: 3 0x0000000000010088 (0x00004537) x10 0x0000000000004000 lui     a0, 0x4
riscv_cpu_read_memory: invalid physical address 0x0000000000004000
hartid=0 priv: 3 exception fault_load, epc 0x000000000001008c
hartid=00           tval 0x0000000000004000
CORE0: 3 0x0000000000010000 (0x020005b7) x11 0x0000000002000000 lui     a1, 0x2000
[error] EMU PC 0000000000010000, DUT PC 000000000001008c
[error] EMU INSN 020005b7, DUT INSN 00056503
[error] EMU WDATA 0000000002000000, DUT WDATA 0000000080000000
[error] EMU MSTATUS a00001800, DUT MSTATUS 00000000
[error] DUT pending exception -1 pending interrupt -1

The above trace is the output of the following command:

make CONFIG=DromajoBoomConfig ENABLE_DROMAJO=1 BINARY=$(path_to_bins)/rv64ui-p-and run-binary

@jerryz123
Copy link
Contributor

jerryz123 commented Jul 11, 2021 via email

@kabylkas
Copy link
Author

Adding the device in dromajo fixed this, thank you!

However, right now, I am hardcoding the address and the return value of the device to mimic BOOM. I was thinking how to make this cleaner.

I noticed that dromajo is referencing dromajo_params.h for some values (see below). I was wondering if this file and the parameters are auto generated? Or it is just a hardcoded values? If the latter is true, maybe I could add more defines to facilitate the above mentioned device?

#ifndef DROMAJO_PARAMS_H
#define DROMAJO_PARAMS_H

#define DROMAJO_RESET_VECTOR "0x10040"
#define DROMAJO_MMIO_START "0x20000"
#define DROMAJO_MMIO_END "0x80000000"
#define DROMAJO_MEM_SIZE "0x100"
#define DROMAJO_PLIC_BASE "0xC000000"
#define DROMAJO_PLIC_SIZE "0x4000000"
#define DROMAJO_CLINT_BASE "0x2000000"
#define DROMAJO_CLINT_SIZE "0x10000"

#endif

@jerryz123
Copy link
Contributor

jerryz123 commented Jul 16, 2021 via email

@kabylkas kabylkas closed this as completed Oct 7, 2021
@kabylkas
Copy link
Author

Several people emailed me asking how to fix this. So for documentation purposes, just leaving a quick note on how I made this work.

To solve this issue, we need to let Dromajo know about the existence of this register. I defined this register by calling the function that is on lines 1147 - 1149 in RISCVMachine *virt_machine_init(const VirtMachineParams *p) function of {DROMAJO_REPO}/src/riscv_machine.cpp:

1142     for (int i = 0; i < s->ncpus; ++i) {
1143         s->cpu_state[i]->physical_addr_len = p->physical_addr_len;
1144     }
1145 
1146 
1147     cpu_register_device(s->mem_map, 0x4000, 8, 0,                    // <--
1148                         bootrom_device_read, bootrom_device_write,   // <--
1149                         DEVIO_SIZE32 | DEVIO_SIZE16 | DEVIO_SIZE8);  // <--
1150 
1151     if (p->mmio_start) {
1152         uint64_t sz = p->mmio_end - p->mmio_start;
1153         cpu_register_device(s->mem_map, p->mmio_start, sz, 0,
1154                             mmio_read, mmio_write, DEVIO_SIZE32 | DEVIO_SIZE16 | DEVIO_SIZE8);
1155     }

Obviously, bootrom_device_read and bootrom_device_write should also be defined in the same file. What I meant by hardcoding in one of the messages above is simply returning 0x80000000 when the register is read:

 132 static uint32_t bootrom_device_read(void *opaque, uint32_t offset, int size_log2)
 133 {
 134     fprintf(dromajo_stderr, "bootrom_device_read: offset=%x size_log2=%d\n", offset, size_log2); 
 135 
 136     return 0x80000000;
 137 }
 138 
 139 static void bootrom_device_write(void *opaque, uint32_t offset, uint32_t val, int size_log2)
 140 {
 141     fprintf(dromajo_stderr, "bootrom_device_write: offset=%x size_log2=%d val=%x\n", offset, size_log2, val);
 142 }

Pretty dirty solution, but should make Dromajo work with BOOM.

@jerryz123
Copy link
Contributor

Can you PR this fix into the dromajo fork we have?

@kabylkas
Copy link
Author

Yeah, I will do it.

@jerryz123
Copy link
Contributor

Thanks

@zbw444
Copy link

zbw444 commented Oct 25, 2021

Excuse me, after fixing the above problems according to your method, I want to ask a new question. Why can I run the program under /isa to pass, but run the program in /benchmarks to make an error? The error message is as follows:

This is displayed when running vvadd.riscv:

3 0x0000000080002436 (0x0000631c) x15 0x0000000000000000 c.ld    a5, 0(a4)
3 0x0000000080002438 (0x0000dffd)                        c.beqz  a5, pc - 2
3 0x0000000080002436 (0x0000631c) x15 0x0000000000000000 c.ld    a5, 0(a4)
[error] EMU PC 0000000080002436, DUT PC 0000000080002436
[error] EMU INSN 0000631c, DUT INSN 0000631c
[error] EMU WDATA 0000000000000000, DUT WDATA 0000000000000001
[error] EMU MSTATUS 8000000a00006080, DUT MSTATUS 00000000
[error] DUT pending exception -1 pending interrupt -1

This is displayed when running spmv.riscv:

3 0x0000000080002542 (0x0000631c) x15 0x0000000000000000 c.ld    a5, 0(a4)
3 0x0000000080002544 (0x0000dffd)                        c.beqz  a5, pc - 2
3 0x0000000080002542 (0x0000631c) x15 0x0000000000000000 c.ld    a5, 0(a4)
3 0x0000000080002544 (0x0000dffd)                        c.beqz  a5, pc - 2
3 0x0000000080002542 (0x0000631c) x15 0x0000000000000000 c.ld    a5, 0(a4)
[error] EMU PC 0000000080002542, DUT PC 0000000080002542
[error] EMU INSN 0000631c, DUT INSN 0000631c
[error] EMU WDATA 0000000000000000, DUT WDATA 0000000000000001
[error] EMU MSTATUS 8000000a00006080, DUT MSTATUS 00000000
[error] DUT pending exception -1 pending interrupt -1

@yswntht yswntht mentioned this issue Mar 20, 2023
Merged
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kabylkas @jerryz123 @zbw444