enable support for loader stubs (via --use-stub arg) #216
Conversation
|
I just checked out this PR because I am hoping stub support will fix the issue that i'm facing; however, when i try to flash with --use-stub I get the following Any thoughts? Also, you might be able to confirm if this would indeed help with the issue i'm trying to resolve to begin with. When I upload using espflash my 'nvs' region seems to get erased (maybe only partially?). This even happens if I only upload the partition table with Partition table I've inspected what espflash is doing internally and it appears that it should be only erasing 0x8000-0x8FFF so I'm not entirely sure whats going on. Biggest difference I've noticed between my previous workflow with using esptool.py is that it uses a stub. EDIT: |
|
@mertzt89 i'm not a maintainer here but i don't think this is related / -suspect- this is worth it's own issue, and it'd be useful to know exactly what module and IC you're using? i'd make sure the partition table has actually been written, and from a quick look it seems like you might need a |
|
@ryankurte I would ignore my 'nvs' issue for now, if I discover more, or need further help I will make a new issue. For reference, when using Disregarding my 'nvs' issue, I was still unable to make use of --use-stub when using this PR with the error shown above. I'm using a generic "rebranded' devkit, but its documentation refers to it as a an esp32doit-devkit-v1 with a 4M flash. |
it appears some boards report SpiAttach failed when flash has already been loaded by the stub
okay i think i found it, on some devices the stub seems to have already performed the |
|
Pulled your latest changes... Indeed changing this so that the result is ignored gets things going for my esp32 as well. espflash/espflash/src/flash_target/esp32.rs Lines 29 to 33 in 312c115 |
|
A new observation with |
no idea, would have to dig into the docs / esptool implementation to see |
|
I took a look at esptool.py and there is a subtle change needed to make things work. There is a second parameter to the change baud message that is set to the initial baud rate for the stub (the current rate before switching). Right now espflash is hardcoded to pass 0 for that parameter. Locally I've hooked up that second parameter and things seem to work now. |
Co-authored-by: Timothy Mertz <mertzt89@gmail.com>
|
good catch, does that latest commit look like you'd expect / work for you? (if not / things are missing you can always attach a |
|
This patch fixes SpiAttach for my esp32 with and without the stub. This reflects the implementation in esptool.py, the main area of focus is in the implementation of encode() for SpiAttachParams. Including this patch, I cannot find any other issues! Great work! I only have the one flavor of esp32 to test on, so it'd probably be good to run my patch (or similar) on yours also. View PatchFrom 57ba43776d73524dfaf34c882c66602c83382585 Mon Sep 17 00:00:00 2001
From: Timothy Mertz <mertzt89@gmail.com>
Date: Mon, 8 Aug 2022 20:21:02 -0500
Subject: [PATCH] Fix SpiAttach when using stub
---
espflash/src/chip/mod.rs | 8 ++++++--
espflash/src/command.rs | 14 ++++++++++++--
espflash/src/flash_target/esp32.rs | 14 +++++++++++---
espflash/src/flasher.rs | 28 +++++++++++++++++++---------
4 files changed, 48 insertions(+), 16 deletions(-)
diff --git a/espflash/src/chip/mod.rs b/espflash/src/chip/mod.rs
index b0b9948..fefb2fe 100644
--- a/espflash/src/chip/mod.rs
+++ b/espflash/src/chip/mod.rs
@@ -299,10 +299,14 @@ impl Chip {
Box::new(RamTarget::new(entry))
}
- pub fn flash_target(&self, spi_params: SpiAttachParams) -> Box<dyn FlashTarget> {
+ pub fn flash_target(
+ &self,
+ spi_params: SpiAttachParams,
+ use_stub: bool,
+ ) -> Box<dyn FlashTarget> {
match self {
Chip::Esp8266 => Box::new(Esp8266Target::new()),
- _ => Box::new(Esp32Target::new(*self, spi_params)),
+ _ => Box::new(Esp32Target::new(*self, spi_params, use_stub)),
}
}
diff --git a/espflash/src/command.rs b/espflash/src/command.rs
index f960596..27fd8e0 100644
--- a/espflash/src/command.rs
+++ b/espflash/src/command.rs
@@ -112,6 +112,9 @@ pub enum Command<'a> {
SpiAttach {
spi_params: SpiAttachParams,
},
+ SpiAttachStub {
+ spi_params: SpiAttachParams,
+ },
ChangeBaud {
/// New baud rate
new_baud: u32,
@@ -150,6 +153,7 @@ impl<'a> Command<'a> {
Command::WriteReg { .. } => CommandType::WriteReg,
Command::ReadReg { .. } => CommandType::ReadReg,
Command::SpiAttach { .. } => CommandType::SpiAttach,
+ Command::SpiAttachStub { .. } => CommandType::SpiAttach,
Command::ChangeBaud { .. } => CommandType::ChangeBaud,
Command::FlashDeflateBegin { .. } => CommandType::FlashDeflateBegin,
Command::FlashDeflateData { .. } => CommandType::FlashDeflateData,
@@ -268,9 +272,15 @@ impl<'a> Command<'a> {
write_basic(writer, &address.to_le_bytes(), 0)?;
}
Command::SpiAttach { spi_params } => {
- write_basic(writer, &spi_params.encode(), 0)?;
+ write_basic(writer, &spi_params.encode(false), 0)?;
}
- Command::ChangeBaud { new_baud, prior_baud } => {
+ Command::SpiAttachStub { spi_params } => {
+ write_basic(writer, &spi_params.encode(true), 0)?;
+ }
+ Command::ChangeBaud {
+ new_baud,
+ prior_baud,
+ } => {
// length
writer.write_all(&(8u16.to_le_bytes()))?;
// checksum
diff --git a/espflash/src/flash_target/esp32.rs b/espflash/src/flash_target/esp32.rs
index 458b445..2c64d02 100644
--- a/espflash/src/flash_target/esp32.rs
+++ b/espflash/src/flash_target/esp32.rs
@@ -13,13 +13,15 @@ use std::io::Write;
pub struct Esp32Target {
chip: Chip,
spi_attach_params: SpiAttachParams,
+ use_stub: bool,
}
impl Esp32Target {
- pub fn new(chip: Chip, spi_attach_params: SpiAttachParams) -> Self {
+ pub fn new(chip: Chip, spi_attach_params: SpiAttachParams, use_stub: bool) -> Self {
Esp32Target {
chip,
spi_attach_params,
+ use_stub,
}
}
}
@@ -27,8 +29,14 @@ impl Esp32Target {
impl FlashTarget for Esp32Target {
fn begin(&mut self, connection: &mut Connection) -> Result<(), Error> {
connection.with_timeout(CommandType::SpiAttach.timeout(), |connection| {
- connection.command(Command::SpiAttach {
- spi_params: self.spi_attach_params,
+ connection.command(if self.use_stub {
+ Command::SpiAttachStub {
+ spi_params: self.spi_attach_params,
+ }
+ } else {
+ Command::SpiAttach {
+ spi_params: self.spi_attach_params,
+ }
})
})?;
diff --git a/espflash/src/flasher.rs b/espflash/src/flasher.rs
index 735d0f4..e7c21d8 100644
--- a/espflash/src/flasher.rs
+++ b/espflash/src/flasher.rs
@@ -138,17 +138,20 @@ impl SpiAttachParams {
}
}
- pub fn encode(self) -> Vec<u8> {
+ pub fn encode(self, stub: bool) -> Vec<u8> {
let packed = ((self.hd as u32) << 24)
| ((self.cs as u32) << 18)
| ((self.d as u32) << 12)
| ((self.q as u32) << 6)
| (self.clk as u32);
- if packed == 0 {
- vec![0; 5]
- } else {
- packed.to_le_bytes().to_vec()
+
+ let mut encoded: Vec<u8> = packed.to_le_bytes().to_vec();
+
+ if !stub {
+ encoded.append(&mut vec![0u8; 4]);
}
+
+ encoded
}
}
@@ -369,7 +372,11 @@ impl Flasher {
_ => {
self.connection
.with_timeout(CommandType::SpiAttach.timeout(), |connection| {
- connection.command(Command::SpiAttach { spi_params })
+ connection.command(if self.use_stub {
+ Command::SpiAttachStub { spi_params }
+ } else {
+ Command::SpiAttach { spi_params }
+ })
})?;
}
}
@@ -537,7 +544,7 @@ impl Flasher {
) -> Result<(), Error> {
let image = ElfFirmwareImage::try_from(elf_data)?;
- let mut target = self.chip.flash_target(self.spi_params);
+ let mut target = self.chip.flash_target(self.spi_params, self.use_stub);
target.begin(&mut self.connection).flashing()?;
let flash_image = self.chip.get_flash_image(
@@ -564,7 +571,7 @@ impl Flasher {
/// Load an bin image to flash at a specific address
pub fn write_bin_to_flash(&mut self, addr: u32, data: &[u8]) -> Result<(), Error> {
- let mut target = self.chip.flash_target(self.spi_params);
+ let mut target = self.chip.flash_target(self.spi_params, self.use_stub);
target.begin(&mut self.connection).flashing()?;
let segment = RomSegment {
addr,
@@ -606,7 +613,10 @@ impl Flasher {
self.connection
.with_timeout(CommandType::ChangeBaud.timeout(), |connection| {
- connection.command(Command::ChangeBaud { new_baud: speed, prior_baud })
+ connection.command(Command::ChangeBaud {
+ new_baud: speed,
+ prior_baud,
+ })
})?;
self.connection.set_baud(speed)?;
std::thread::sleep(Duration::from_secs_f32(0.05));
--
2.37.1
|
|
I pulled down the latest and everything works for me! |
|
I just noticed that the clippy check on the previous patch failed, it will likely still fail. My patch fixed the rustfmt issue. |
I've been meaning to add this for awhile and have just kept putting it off. I'll start adding in some logging to improve the situation. |
i needed logging for diagnostics so, the basics are now also in this PR |
|
You can run this locally to run the same check. Also you can add |
|
This seems to be working for the ESP32, but I have been unable to get flashing to work when using the ESP32-C3, ESP32-S2, or ESP32-S3. For each chip I get the following error: The I will try to dig into the code and find the problems, so I'll report back if I figure it out. |
|
I just reviewed esptool.py again, It might have to do with the fact that they subclassed the stubloader for each IC to tweak a few parameters or perform a few IC specific steps. I have an 8266, s2, s3, and c3 on they way, I should have them this weekend, I am happy to help debug. |
|
I received a couple of my new devices today and this seems to do the trick! From da2ba994d5d240bd2a88eff10281faee980d5d79 Mon Sep 17 00:00:00 2001
From: Timothy Mertz <mertzt89@gmail.com>
Date: Sun, 14 Aug 2022 23:45:01 -0500
Subject: [PATCH] Don't use encryption when using stub
---
espflash/src/flash_target/esp32.rs | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/espflash/src/flash_target/esp32.rs b/espflash/src/flash_target/esp32.rs
index 2c64d02..462d7b8 100644
--- a/espflash/src/flash_target/esp32.rs
+++ b/espflash/src/flash_target/esp32.rs
@@ -108,7 +108,7 @@ impl FlashTarget for Esp32Target {
blocks: block_count as u32,
block_size: FLASH_WRITE_SIZE as u32,
offset: addr,
- supports_encryption: self.chip != Chip::Esp32,
+ supports_encryption: self.chip != Chip::Esp32 && !self.use_stub,
})?;
Ok(())
},
--
2.37.1I assume you are using a UART bridge with your S2 based on the fact that you have received the same errors. It turns out that espflash does not properly support communicating with an S2 with CDC enabled in the bootloader. I have it basically working but its based on this work so I'll wait for this to be merged. |
With respect to the ReadReg error, I would see #217 I think it is actually the root cause. |
Co-authored-by: Timothy Mertz <mertzt89@gmail.com>
|
I tested and merged #217, everything looked fine there. Unfortunately when I rebase this branch on λ cargo espflash board-info --use-stub
Detected 2 serial ports. Ports which match a known common dev board are highlighted.
Serial port: /dev/cu.usbserial-1440
Connecting...
Using flash stub
Error: espflash::timeout
× Error while connecting to device
╰─▶ Timeout while running command |
Which device are you trying to do it with? If the S2, with native USB (CDC) or UART? |
|
I'm using the UART for all boards. I tried again on an ESP32-C3 using the USB Serial JTAG and I see the same error. |
|
I'll try with my C3 tonight I just got my hands on it, was the last one I was waiting on. |
|
I locally rebased this branch on master and tested each of my devices via UART (ESP32, ESP32-S2, ESP32-S3, and ESP32-C3) and I did not have any issues. I used the following to test each device in a loop. while true; do espflash --use-stub board-info; if [[ $? -ne 0 ]]; then break; fi; doneI'm not sure why we are having different results. |
|
The Switching to |
|
Ah! Yeah i see the problem now. Looking in to it. Definitely didn't notice the arg placement being important. |
|
Oh! I know the issue! I'll put a patch up |
|
This fixed things for me From c80d6b22b01e22ec9ab94b30b2d198d2b67624c6 Mon Sep 17 00:00:00 2001
From: Timothy Mertz <mertzt89@gmail.com>
Date: Wed, 17 Aug 2022 12:33:27 -0500
Subject: [PATCH] Look for stub handshake
---
espflash/src/error.rs | 2 ++
espflash/src/flash_target/ram.rs | 7 +++----
espflash/src/flasher.rs | 8 ++++++--
3 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/espflash/src/error.rs b/espflash/src/error.rs
index 6933f69..8d0d61a 100644
--- a/espflash/src/error.rs
+++ b/espflash/src/error.rs
@@ -152,6 +152,8 @@ pub enum ConnectionError {
help("Try hard-resetting the device and try again, if the error persists your rom might be corrupted")
)]
OverSizedPacket,
+ #[error("Invalid stub handshake response received")]
+ InvalidStubHandshake,
}
#[derive(Debug, Default, Clone)]
diff --git a/espflash/src/flash_target/ram.rs b/espflash/src/flash_target/ram.rs
index f0f601a..f3edb10 100644
--- a/espflash/src/flash_target/ram.rs
+++ b/espflash/src/flash_target/ram.rs
@@ -67,13 +67,12 @@ impl FlashTarget for RamTarget {
if reboot {
let entry = self.entry.unwrap_or_default();
connection.with_timeout(CommandType::MemEnd.timeout(), |connection| {
- connection.write_command(Command::MemEnd {
+ connection.command(Command::MemEnd {
no_entry: entry == 0,
entry,
})
- })
- } else {
- Ok(())
+ })?;
}
+ Ok(())
}
}
diff --git a/espflash/src/flasher.rs b/espflash/src/flasher.rs
index cebf867..719e815 100644
--- a/espflash/src/flasher.rs
+++ b/espflash/src/flasher.rs
@@ -26,6 +26,8 @@ const FLASH_SECTORS_PER_BLOCK: usize = FLASH_SECTOR_SIZE / FLASH_BLOCK_SIZE;
// register used for chip detect
const CHIP_DETECT_MAGIC_REG_ADDR: u32 = 0x40001000;
+const EXPECTED_STUB_HANDSHAKE: &str = "OHAI";
+
#[derive(Clone, Copy, Debug, Eq, PartialEq, Display, EnumVariantNames)]
#[repr(u8)]
pub enum FlashSize {
@@ -289,8 +291,10 @@ impl Flasher {
debug!("Stub written...");
- // Re-sync connection
- self.connection.sync()?;
+ match self.connection.read(EXPECTED_STUB_HANDSHAKE.len())? {
+ Some(resp) if resp == EXPECTED_STUB_HANDSHAKE.as_bytes() => Ok(()),
+ _ => Err(Error::Connection(ConnectionError::InvalidStubHandshake)),
+ }?;
// Re-detect chip to check stub is up
let magic = self.connection.read_reg(CHIP_DETECT_MAGIC_REG_ADDR)?;
--
2.37.2
|
|
Awesome, that seems to have resolved the problem! Thanks for digging into that, I think once that patch gets applied to this PR we're probably good to go! |
Co-authored-by: Timothy Mertz <mertzt89@gmail.com>
adds a rough impl for loading flash stubs into ram towards #215 (and esp-rs/esp-idf-template#48), enabled with the
--use-stubargument so this is a strictly opt-in addition and doesn't change existing behaviour.(for some reason
ConnectOptsis part ofmain::OptsandSubcommand::(BoardInfo|SerialMonitor)which can be a bit confusing / seems like it should only be in the root?)tested
board-infoand flashing with on myesp32-pico-v3:(as an aside, it'd be really nice to have a logging framework in there for debugging etc.)