Done.txt

Convert to VS2012

Added PDB output:
Anyway, it would be possible to update ACME to output a full debug map for each memory location with source file and line number.
It would also be possible to accurately save out what labels and bytes for what memory are where.
Including zone information would allow local labels to be displayed first within the zone scope.

Then use the remote debugger to display nice binary disassembly and source code mixed in.

Implementation notes:
*Functions like "real_output" can be used to provide extensive debug information.
For example when "*write_ptr++" is used.
*Look for Parse_and_close_file, tends to be used for source input files.
Look at Input_new_file it has Input_now->line_number and Input_now->original_filename already setup

*Implement the file write for PDB_filename in ACME_finalize after it does vicelabeldump_filename
	In PDBSave()

*Output format:
FILES:<dec num for the number of files used>
<index0>:<filename>
...
ADDRS:<dec num of the addresses used>
$<hex num address>:<dec zone>:<dec index into FILES>:<dec line number>
...
LABELS:<dec num of the labels>
$<hex num>:<dec zone>:<text label>:<1/0 = USED>:<1/0 = ADDR>
...



Rebuilt the project from scratch to avoid problems with the conversion from the DSW/DSP

Address needs zone list when it changes


Improved PO: "!binary" now has "interleave" parameter.



* Added VICEPDBMonitor
VICE C64 remote debugging
Using VICE x64.exe -remotemonitor
Then telnet localhost 6510
It's possible to break into the execution by sending a character. For example to get new "m 0000" memory while it's running code. Or to break execution for step with 'z'.




* When assembling, if a variable is forward referenced (unknown) for !for or other macros, then assume it is 1 on the first pass, then if the value changes in later passes, re-visit the code with the new values.
	If the value is unchanged during a pass and still unknown then report it cannot be resolved.
	Or if the variable changes too many times during passes, report it cannot converge to a stable value.
	* See "TestForForwardReference.a" for an example of convergence assembly
	* ALU_defined_int() could try to use ALU_any_int() and note the expression used with the zone+file+line and if the expression is unknown too many times then give up.
		Increment pass_undefined_count when the expression is unknown or changes from previous value
		Label_set_value() will need an override that ignores any redefinition when an undefined label used by !for is encountered
	* To help give a hint where the new forward reference attempts were used, a new serious error was created "Value not defined. Even after several attempts."
	* -v4 will output full debug for detection of changing calculation values to allow these to be efficiently traced



* Might need a !PO to stack and set the verbosity level, then pop the vebosity level:
	To mitigate reporting of this where !for is inside a macro and the values legitimately change: asm/RoadObject_Macro.a(27) : Warning (Macro RoadObject): Detected result change: 143: height:asm/RoadObject_Macro.a:27 : From 17 to 19
	Maybe it's to do with the macro usage not being in a separate zone?
	* Mitigated without verbosity push, but instead added the zone to the calculation history test




* --lib will not allow command line files to be found, expand the file search to use any --lib paths



* !to seems to be outputting the file to the wrong directory (Scroller not Tusari2) when building Tusari2?




* Add "!previouscontext" which will output debug for the previous context level not the current source file/line
	Probably add it near: PREDEFNODE("sal",	PO_sal),	// MPi: Added
	Altering how PDBAddFileLineToAddr is called
	Might be a good place to store nested file/line/zone information:	void Macro_parse_call(void)
	Perhpas "Macro_parse_call" can backup these: Input_now->original_filename , Input_now->line_number , Section_now->zone
	Then if there is a "!previouscontext" then the call to PDBAddFileLineToAddr will use the most recent backups
	Added int previouscontext_enable



* Sorted by address labels output
	void SortFile( const char *filename )



* Produce concise ranges of free memory in the PDB file
	e.g. <start>-<end>:<size>
		FREEMAP:4
		$0-$3ff:$400
		$452-$45e:$d
		$504-$fff:$afc
		$1004-$ffff:$effc




* !swizzle address , 256 , 3
	This will take the 256*3 bytes of data at address and apply a swizzle, which converts row oriented data into columns, or vice versa.
	For example if there was data compresed of 256 repeating patterns of: !word value !by value : so 256 x 3 bytes
	It will convert the data into:
		256 bytes of < bytes (lo !word)
		256 bytes of > bytes (hi !word)
		256 bytes of byte values



* Implement these extensions to the formula value parser
	* [d LABEL or any formula]
	* [f "filename" , LABEL or any formula]
	* !set LABEL = [d LABEL or hex value for ADDRESS]
		Will set label equal to the unsigned byte value of the assembled data
		This is useful for importing binart data with !bin and then processing the data to produce label offsets into it.
	* !set LABEL = [f "filename" , offset]
		Will set label equal to the unsigned byte value at file offset for filename.
		This is useful for processing the file structure of a binary file to produce label values.
	> PO_set
	> ALU_any_result
	> parse_expression
	>> Add test to TestForForwardReference.a
	> Can treat these like: case '(':// left parenthesis
	// Done: Need to read byte at memory
		real_output?
		write_ptr?
		Might need something to get a byte, like real_output but gets not stores
			>>getByteAtAddr
	* Surely this would have been easier with peek()/poke()/filebyte() etc?
		Like: PREDEFNODE(s_sin,	&ops_sin),
		Or: PREDEFNODE("int",	&ops_int),
	* I think it's going to be easier to enhance this to add string function parameter parsing:
			if(Tree_easy_scan(function_tree, &node_body, function_dyna_buf))
			PUSH_OPERATOR((op_t*) node_body);
			Use Input_read_filename() like for PO_binary()?
		filebyte() works
		peekbyte() works
		!disablesegmentcheck
		!enablesegmentcheck




* Scripting options?
	* Using a PO ! like:
		* !scriptsimple
		* !scriptpython
			https://docs.python.org/3/extending/embedding.html
			https://docs.python.org/3/extending/embedding.html#extending-embedded-python
			https://docs.python.org/3/c-api/index.html#c-api-index
			Extension function example: https://docs.python.org/3/extending/extending.html#a-simple-example
				Could create: acme.source("text")
					acme.binary(bytearray)
						https://www.geeksforgeeks.org/python-bytearray-function/
					acme.setPC(address)
					acme.addLabel(text)
					acme.newZone()
						>> PO_zone
							>> Parse_optional_block?
					acme.startZone()
					acme.endZone()
					* The source added via this function can be processed like !source ?
						>> PO_source
							>> Parse_and_close_file
								>> new_input.source_is_ram = TRUE;
			"%LOCALAPPDATA%\Programs\Python\Python39\include"
				Python.h
			"%LOCALAPPDATA%\Programs\Python\Python39\Libs"
				python39.lib
			>> Can also use some of the code in PO_for for define an inline pyton code block
				loop_body = Input_skip_or_store_block(TRUE)
			** Two options:
				!scriptpython {
				# Some python code inside braces
				}
				!scriptpython "filename.py"
	* Key functionality would be:
		* Read currently defined labels, and write new values
		* Read and write assembled memory
		* Inject new text assembly to be assembled outside of the script
		* Execute scripts from external files would probably be the most common use case.
			It might make sense to export the labels to a python format, ready to be imported?
				It would be slow, but easier than relying on durect to Python memory injection.
	poArgs = [0, 1234, "hello"]
	# address = (bank, address)
	addressToLabels = {
		(0, 1234): [
			{"name": ".aLabel",
				"zone": 1,
				"used": True
				},
			{"name": "global",
				"zone": 0}
		],
		(0, 1235): {"just this one"}
	}

	labelToAddresses = {
		".aLabel": [(0, 1234)],
		".l1": [(1, 134), (0, 436)]
	}
	>> Note: c:\users\martin piper\appdata\local\programs\python\python39\include\pyport.h
		Needed:
			// NOTE: Wrong header
			//#include <inttypes.h>
			#include <stdint.h>
		>> LINK : fatal error LNK1104: cannot open file 'python39_d.lib'
			C:\Users\Martin Piper\AppData\Local\Programs\Python\Python39\libs
				Needed to copy "python39.lib" to "python39_d.lib"
			>> copy %LOCALAPPDATA%\Programs\Python\Python39-32\Libs\python39.lib %LOCALAPPDATA%\Programs\Python\Python39-32\Libs\python39_d.lib
	>> The EXE type needed to be changed to x64 and python.dll is x64
	* Done: For anonymous errors like: Error - File TestPython.py, line 1 (Zone <untitled>): Value not defined.
		These need to emit the problematic line which includes the label etc...
			throw_message can output the information
			However Parse_until_eob_or_eof will need to "save" the last processed line
				>> ALU_any_result
					>> parse_expression
						>> Perhaps this can save the last expression that was parsed?
							Which can then be output
	* Done: !scriptpythonfile "TestPython.py" , <parameter 1> , <parameter n>
	* Done: !scriptpythoninline <parameter 1> , <parameter n> {
		For these can use the !for model
	* Done: Automatically include the "import acme", then parameters
	* TODO: Automatically add labels
	* Done: Change back to use x86 build from: https://www.python.org/downloads/release/python-390/
		https://www.python.org/ftp/python/3.9.0/python-3.9.0.exe
			C:\Users\Martin Piper\AppData\Local\Programs\Python\Python39-32




* It might be possible to find the current pyton file and line being executed and pass that to the assembly PDB information
	>> Note: This line seems to be accurate and includes the common header we add on
		> This will always be "<string>" because we use PyRun_SimpleString()
	This means when it's <string> the "name" (filename) passed in to RunScript_Python() plus a line number offset, can be used.
		When fullSource is being composed, the line number offset can be further adjusted by the size of the source preamble that does the import and defines acmeParameters etc.
		> Input_now should be updated with the filename and any adjusted line number
	Use command line parameters: -v9 --pdb target\TestPython.pdb --labeldump target\TestPython.lbl TestPython.a




* Definitely need acme.byte(value) to output a raw byte at the current PC, skipping all the processing by acme.source("	!by 12") etc
	PO_08
	acme.byte([values...])
	or even
	acme.byte(val1, val2, etc)
	acme.word()
	etc




* Think about:
	sys.path.insert(0, '../../C64/stdlib')
	Could this be added automatically based on the lib path?




* acme_bytestr() will need proper tests for the entire number range, with and without " and ' characters
	Due to needing this kind of test: if (strncmp(command , "b'" , 2) == 0 || strncmp(command , "b\"" , 2) == 0)




* PDB address information is wrong when using !pseudopc
	It is outputting the address where it is in the assembled memory, not the intended address
	> Probably add some detection to PO_pseudopc to add an address mapping to PDB (like PDBAddFileLineToAddr )
		>> Useful globals for PDBAddFileLineToAddr
			current_offset (but needs new signed offset)
			uses_pseudo_pc
		Currently the PDB debugger bodges APU support by looking for mAPUCode_Start
		So when the PDB output stage happens, any address mappings can be added
		> Clear the address mapping table each pass. Only the last pass mapping table is used, basically.
	>> The values in the "LABELS:" section are OK, it's just the values in the "ADDRS:" section
		Yes the code is stored in the RAM, so it still needs to original PDB info, however the PDB info also needs a duplicate to account for the destination address (and device)
	> Note code that updates these: uses_pseudo_pc
		current_offset
		PO_pseudopc
	* Temporarily removing "mDriveCode_Start" and "mDriveCode_StartReal" initialisation shows the new pseudo PC output works.




* Remove duplicates from the ADDRS: list





* It would be better to include some kind of device ID value for PDB debugging, either in the address or in the zone.
	This is because currently APU code, or drive code, addresses can overlap with C64 addresses in the memory map
	The current PDB debuggder handles APU code view by bodging the address by looking for "APUCode_Start" and setting mAPUCode_Start
	Perhaps the top address bits would be better??
		The request for PDB information can then add in the requesting device ID

	A new pseudo op comand to set the current device ID for the code being assembled up until the end of the block?
	>> Adding (logical ORing) a number into the top 8 bits of the zone would probably be best.
		C: = 0x00
		8: = 0x08
		APU: = 0x40
	>> Then in VICEPDBMonitor all usage of getAddrInfoForAddr() can be updated to include the device

	> Could use code like PO_zone
	* Hmm perhaps expanding !zone would be the bext option...
		Like: !zone C:
		>> New command: !device which copies !zone mostly
	* TODO: Look for " << 24" in relation to device and make this a common function?

	* Done: TestACMe.a shows a bug where the same !pseudopc address does not output multiple entries for that address in the PDB
	** Done: Separate device in the PDB, don't merge with zone.
	mAddrInfoByAddr is going to have be flattened according to the current filter...
		When UpdateLabels does its work?
	>> refreshContextListForAddress()
		Actually, this only need a multimap for each address in the device type...
	>> checkForChangedContext()
	** addrInfo.mContext is null, it needs to be populated much earlier... At the end of the PDB load basically...





* Zone overlaps another zone should be an error, not a warning.
	Default to error, option to make it a warning.
		--warnzoneoverlap
		OPTION_WARNZONEOVERLAP, like OPTION_MSVC




* Add a !PO to disable and enable: Error (Zone <untitled>): Label already defined. : start
	C:\work\C64\IRQDisk\SmallestBoot.a
		Avoid the need for: !set .start = *
	--msvc -v9 -f cbm -o c:\temp\t.prg TestLabelRedefinition.a
	Could increment gLabel_set_value_changed_allowed
	!labelredefine
	!labelredefineend