Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement user-space QEMU ASAN #1806

Merged
merged 23 commits into from
Apr 9, 2024
Merged

Implement user-space QEMU ASAN #1806

merged 23 commits into from
Apr 9, 2024

Conversation

WorksButNotTested
Copy link
Collaborator

@WorksButNotTested WorksButNotTested commented Jan 18, 2024 •
edited
Loading

Seems that QEMU now no longer baulks when attempting to allocate large MAP_NORESERVE ranges in the guest. This means that it is now possible to keep the shadow ranges in the host rather than the guest. Accordingly, it means that TCG can be used to test the shadow maps for normal loads/stores and we don't need to make any phony syscall or similar to handle any interactions in the host (including any g2h translations).

Unlike the first iteration, we now use a modified libqasan.so rather than a stock clang or gcc libasan.so in the guest. The asan library in the guest must update the shadow map and so we need a means to avoid instrumenting these updates (otherwise the corresponding access falls within the shadow gap). Instrumenting the instrumentation is a bad idea!

A simple approach to avoiding this problem, is to simply skip emitting the TCG ASAN code when instrumenting the asan library. However, the stock clang and gcc libraries modify the shadow map indirectly through libc calls such as memset. Therefore we favour a modified libqasan.so, renamed as libgasan.so which performs these updates directly.

@WorksButNotTested WorksButNotTested force-pushed the asan branch 2 times, most recently from 99ff861 to 6b29b9a Compare January 18, 2024 19:20
@WorksButNotTested WorksButNotTested force-pushed the asan branch 8 times, most recently from ba509df to 3b3fd97 Compare February 6, 2024 18:41
@domenukk
Copy link
Member

thanks!
This doesn't seem to build (anymore?)
Also, it may need a ./script/format.sh (or whatever the name was)

@WorksButNotTested
Copy link
Collaborator Author

This branch also requires a modified qemu-libafl-bridge, here, so that may account for your build errors?
https://github.com/WorksButNotTested/qemu-libafl-bridge/tree/asan

However, in the interim, it seems that this commit has caused a regression for me?
WorksButNotTested/qemu-libafl-bridge@0c51581
https://discord.com/channels/908658106072969256/1181604592769839186/1205167820778901594

@domenukk
Copy link
Member

However, in the interim, it seems that this commit has caused a regression for me? WorksButNotTested/qemu-libafl-bridge@0c51581 https://discord.com/channels/908658106072969256/1181604592769839186/1205167820778901594

What's the regression? Should we revert this commit?

@WorksButNotTested
Copy link
Collaborator Author

WorksButNotTested commented Feb 16, 2024 •
edited
Loading

It seems that mappings are omitted from the output. In particular if you enumerate them twice you seem to get different results.

I put the details in the discord I think. What was the purpose of the commit?

@domenukk
Copy link
Member

@cube0x8 would know better

@andreafioraldi
Copy link
Member

andreafioraldi commented Feb 16, 2024 via email

@cube0x8
Copy link
Contributor

cube0x8 commented Feb 16, 2024 •
edited
Loading

@WorksButNotTested the interval tree's root of the mappings is of type IntervalTreeRoot and is not big enough to be casted to a MapInfo. So, this memory access is going to be out-of-bound when performed on the tree's root. It's very hard you're going to get a h2g_valid address in this way, so I guess it does not represent a real problem. I was investigating a problem in my fuzzer and this bug was blocking me from using qemu-libafl-bridge compiled with ASAN, but if it's broken I guess you can revert it.

@WorksButNotTested
Copy link
Collaborator Author

@WorksButNotTested the interval tree's root of the mappings is of type IntervalTreeRoot and is not big enough to be casted to a MapInfo. So, this memory access is going to be out-of-bound when performed on the tree's root. It's very hard you're going to get a h2g_valid address in this way, so I guess it does not represent a real problem. I was investigating a problem in my fuzzer and this bug was blocking me from using qemu-libafl-bridge compiled with ASAN, but if it's broken I guess you can revert it.

If you are able to reproduce the problems I'm having (hopefully should be fairly straightforward if you follow the steps in the discord), then you could always try and find the root cause if you'd rather fix than revert?

@cube0x8
Copy link
Contributor

cube0x8 commented Feb 16, 2024

Ok I'll take a look and see how I can fix it

@domenukk
Copy link
Member

domenukk commented Feb 16, 2024 •
edited
Loading

Don't think you have access to this discord, copying it here for visibility:

"Seems that mappings are going missing?"
mappings.txt

This is the code that is running: https://github.com/WorksButNotTested/LibAFL/blob/f69f38ad439075d19a7f83f8f4274df25c4638b1/libafl_qemu/src/asan_guest.rs#L155

First I print direct from the iterator, then again after I have mapped them to a different type.

mapping: 0x000000007fff8000-0x000000008fff7000 prw- 0x0000000000000000                                                                                                                                             
mapping: 0x000002008fff7000-0x000010007fff8000 prw- 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaaaaab000-0x00002aaaaaaac000 p--- 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaaaaac000-0x00002aaaab2ac000 prw- 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaab2ac000-0x00002aaaab2e3000 pr-- 0x0000000000000000 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2                                                                                              
mapping: 0x00002aaaab2e3000-0x00002aaaab2e4000 p--- 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaab2e4000-0x00002aaaab2e6000 pr-- 0x0000000000037000 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2                                                                                              
mapping: 0x00002aaaab2e6000-0x00002aaaab2e8000 prw- 0x0000000000039000 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2                                                                                              
mapping: 0x00002aaaab2e8000-0x00002aaaab2e9000 pr-x 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaab2e9000-0x00002aaaab2ed000 prw- 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaab2ed000-0x00002aaaab2fa000 pr-- 0x0000000000000000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/release/libgasan.so                                                                
mapping: 0x00002aaaab2fa000-0x00002aaaab2fb000 pr-- 0x000000000000c000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/release/libgasan.so                                                                
mapping: 0x00002aaaab2fb000-0x00002aaaab2fc000 prw- 0x000000000000d000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/release/libgasan.so                                                                
mapping: 0x00002aaaab2fc000-0x00002aaaab2fe000 prw- 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaab316000-0x00002aaaab530000 pr-- 0x0000000000000000 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30                                                                                               
mapping: 0x00002aaaab530000-0x00002aaaab531000 p--- 0x000000000021a000 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30                                                                                               
mapping: 0x00002aaaab531000-0x00002aaaab53c000 pr-- 0x000000000021a000 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30                                                                                               
mapping: 0x00002aaaab53c000-0x00002aaaab53f000 prw- 0x0000000000225000 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30                                                                                               
mapping: 0x00002aaaab53f000-0x00002aaaab542000 prw- 0x0000000000000000                                                                                                                                             
mapping: 0x00002aaaab542000-0x00002aaaab627000 pr-- 0x0000000000000000 /usr/lib/x86_64-linux-gnu/libm.so.6                                                                                                         
mapping: 0x00002aaaab627000-0x00002aaaab628000 pr-- 0x00000000000e4000 /usr/lib/x86_64-linux-gnu/libm.so.6                                                                                                         mapping: 0x00002aaaab628000-0x00002aaaab629000 prw- 0x00000000000e5000 /usr/lib/x86_64-linux-gnu/libm.so.6                                                                                                         mapping: 0x00002aaaab629000-0x00002aaaab647000 pr-- 0x0000000000000000 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1                                                                                                     
mapping: 0x00002aaaab647000-0x00002aaaab648000 pr-- 0x000000000001d000 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1                                                                                                     
mapping: 0x00002aaaab648000-0x00002aaaab649000 prw- 0x000000000001e000 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1                                                                                                     
mapping: 0x00002aaaab649000-0x00002aaaab85e000 pr-- 0x0000000000000000 /usr/lib/x86_64-linux-gnu/libc.so.6                                                                                                         mapping: 0x00002aaaab85e000-0x00002aaaab85f000 p--- 0x0000000000215000 /usr/lib/x86_64-linux-gnu/libc.so.6                                                                                                         mapping: 0x00002aaaab85f000-0x00002aaaab863000 pr-- 0x0000000000215000 /usr/lib/x86_64-linux-gnu/libc.so.6                                                                                                         
mapping: 0x00002aaaab863000-0x00002aaaab865000 prw- 0x0000000000219000 /usr/lib/x86_64-linux-gnu/libc.so.6                                                                                                         
mapping: 0x00002aaaab865000-0x00002aaaab877000 prw- 0x0000000000000000                                                                                                                                             
mapping: 0x0000555555556000-0x00005555555b1000 pr-- 0x0000000000000000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/libpng-harness-x86_64                                                              
mapping: 0x00005555555b1000-0x00005555555b2000 pr-- 0x000000000005a000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/libpng-harness-x86_64                                                              
mapping: 0x00005555555b2000-0x00005555555b3000 prw- 0x000000000005b000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/libpng-harness-x86_64                                                              
guest mapping: 0x000000007fff8000-0x000000008fff7000                                                                                                                                                               
guest mapping: 0x00002aaaaaaab000-0x00002aaaaaaac000                                                                                                                                                               
guest mapping: 0x00002aaaaaaac000-0x00002aaaab2ac000                                                                                                                                                               
guest mapping: 0x00002aaaab2ac000-0x00002aaaab2e3000 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2                                                                                                                
guest mapping: 0x00002aaaab2e4000-0x00002aaaab2e6000 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2                                                                                                                
guest mapping: 0x00002aaaab2e8000-0x00002aaaab2e9000                                                                                                                                                               
guest mapping: 0x00002aaaab2e9000-0x00002aaaab2ed000                                                                                                                                                               
guest mapping: 0x00002aaaab2ed000-0x00002aaaab2fa000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/release/libgasan.so                                                                                  
guest mapping: 0x00002aaaab2fb000-0x00002aaaab2fc000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/release/libgasan.so                                                                                  
guest mapping: 0x00002aaaab316000-0x00002aaaab530000 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30                                                                                                                 
guest mapping: 0x00002aaaab531000-0x00002aaaab53c000 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30                                                                                                                 
guest mapping: 0x00002aaaab53f000-0x00002aaaab542000                                                                                                                                                               
guest mapping: 0x00002aaaab542000-0x00002aaaab627000 /usr/lib/x86_64-linux-gnu/libm.so.6                                                                                                                           
guest mapping: 0x00002aaaab628000-0x00002aaaab629000 /usr/lib/x86_64-linux-gnu/libm.so.6                                                                                                                           
guest mapping: 0x00002aaaab647000-0x00002aaaab648000 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1                                                                                                                       
guest mapping: 0x00002aaaab649000-0x00002aaaab85e000 /usr/lib/x86_64-linux-gnu/libc.so.6                                                                                                                           
guest mapping: 0x00002aaaab85f000-0x00002aaaab863000 /usr/lib/x86_64-linux-gnu/libc.so.6                                                                                                                           
guest mapping: 0x00002aaaab865000-0x00002aaaab877000                                                                                                                                                               
guest mapping: 0x0000555555556000-0x00005555555b1000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/libpng-harness-x86_64                                                                                
guest mapping: 0x00005555555b2000-0x00005555555b3000 /home/jon/git/LibAFL/fuzzers/qemu_launcher/target/x86_64/libpng-harness-x86_64

This was referenced Feb 16, 2024
Merged
Merged
andreafioraldi
andreafioraldi previously requested changes Mar 30, 2024
View reviewed changes
Copy link
Member

@andreafioraldi andreafioraldi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WorksButNotTested I merged you qemu bride PR, can you merge the conflicts after my fixes and tell if the state of the ASAN implementation is good to be merged?

@andreafioraldi andreafioraldi requested a review from rmalmain March 30, 2024 18:44
@andreafioraldi andreafioraldi enabled auto-merge (squash) April 9, 2024 16:50
@domenukk domenukk dismissed andreafioraldi’s stale review April 9, 2024 23:11

Think this is done

@andreafioraldi andreafioraldi merged commit 374f873 into AFLplusplus:main Apr 9, 2024
78 checks passed
@WorksButNotTested WorksButNotTested deleted the asan branch April 10, 2024 08:44
rmalmain pushed a commit that referenced this pull request Apr 10, 2024
@WorksButNotTested @domenukk
@andreafioraldi @rmalmain
Implement user-space QEMU ASAN (#1806)
43dd353 
* Implement user-space QEMU ASAN

* Fix wrong cfgs

* fmt

* merge conflicts in libafl qemu

* A few more fixes to qemu_launcher

* Change commit of qemu-libafl-bridge

* Fix clippy in qemu_launcher

* Fix commit id again

* Empty commit to trigger CI

* Fix path to fuzzer for test in qemu_launcher?

* Revert location of target binary and show the full error log from qemu_launcher test

* Appease the clippy gods

* Empty

* Fix format

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andreafioraldi andreafioraldi andreafioraldi left review comments

@rmalmain rmalmain Awaiting requested review from rmalmain

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@WorksButNotTested @domenukk @andreafioraldi @cube0x8