Private/edvint/static #1

Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

…jameshen/host_https_only CP-40752 Added CLI functionality for pool level setter and getter (https_only)

Revert "Add a fallback system for auth files belonging to RPMs"

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This means that now xensource.log will contain the reason that produced these exceptions Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The optional parameter for firmware served no purpose, and forced the addition of a unit parameter, remove both. This forces the unit-tests to use a particular firmware to run all tests, use Bios as the default. The firmware value is extracted in preparation of boot blocking depending on the firmware and the presence of vTPM Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The Physical Presence Interface part of the vTPM specification is only implemented for UEFI guests, other functionality may be missing. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This makes it easy to track and link the removal of the function in the quality gate to promotion of the feature to a stable one. Also make raising the exception configurable so developers can easily disable the check and test changes around the functionality without having to recompile xapi. This is done by writing xapiflags=-ignore-vtpm-unimplemented in /etc/sysconfig/xapi and running xe-toolstack-restart Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

We're unable to serialize the data because the field for the contents is not exposed in the API and it's based on a secret, which can be dangerous once it's been implemented. Exports are exposed using an HTTP endpoint, this means it's an indirect operation and that other operations that use the feature will fail in extraneous ways in a non-instantaneous way. To avoid this the two methods that use it in xapi are changed as well (vm-export and VM cross-pool migrations). This makes the failure immediate and clear. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

…service-failures xenopsd/xc: Print all information in Service_failed exceptions

…-tpm CP-39744, CA-370858: Block actions depending on unimplemented VTPM functionality

Add HTTPS support to Open_uri

…fig_script helper Signed-off-by: jameshensmancitrix <james.hensman@citrix.com>

Signed-off-by: Konstantina Chremmou <konstantina.chremmou@citrix.com>

Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

…rewall-1-INPUT chain Signed-off-by: jameshensmancitrix <james.hensman@citrix.com>

The two new libs need to be explicitly mentioned. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…jameshen/host_https_only CP40753 Firewall updates when Host.set_https_only is called

Changed dep for Alpine to correct xxhash package. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

…and false if it is closed, this is captured in set_https_only to update the DB based on the tate of the network not the requested setting should there be a failure Signed-off-by: jameshensmancitrix <james.hensman@citrix.com>

Qemu sends a signal to swtpm to cleanly shut down the guest, this needs Qemu being shut down last Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

…jameshen/host_https_only CP-40754 Update host.https_only field in dbsync based on firewall state

For the time being, only intra-pool migrations will have certificate checking turned on. The new parameter informs xenopsd about the choice. This only matters if an https URL is specified. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

This switches the xenopsd-to-xenopsd connection over to HTTPS, if enabled in the config file (currently off by default). Socket keepalives do not work when stunnel is used, as the given fd is the local connection to stunnel, and are not set in HTTPS mode. This is fine, because the stunnel client is already set up with keepalives. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

The host.migrate_receive call returns URLs for the sending host to use in its VM.migrate_send call. HTTPS URLs are returned based on the value of the config option. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…ilod/sdk CP-40402: Move C# and Powershell SDK Generation to .NET Standard 2.0 / .NET 6.0

…last

The function is question redirects a local storage call to another host if the current host does not have access to the SR. This involves rewriting a localhost HTTP URL to a remote URL, which must be an HTTPS URL if storage migration is configured to use HTTPS. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Storage calls that are handled by the local host should always use HTTP, and not set up a connecting through stunnel. The SM URL that is passed to VM.migrate_send in the `dest` parameter, which comes from the return value of a call to host.migrate_receive, may be an HTTPS URL to the localhost, which needs to be rewritten upon receiving it. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

When starting a mirroring process for a disk as part of a storage migration, xapi establishes the connection to the destination, but hands it over to tapdisk to do the actual mirroring over NBD. It is crucial that xapi just hands over the file descriptor and then continues with other business, without waiting for the connection to finish. This is how it works for TCP connections now. When switching to TLS connections, xapi starts an stunnel process as part of the connection setup, and hands over the stunnel fd to tapdisk. By default, this functionality then waits for stunnel to finish after the connection is eventually broken, thus introducing the unwanted blocking. We fix this by telling stunnel to disconnect, but not wait for this to actually happen. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Add shorter synonyms for creating paths and writing files Add empty mli since it should not export any functions Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

An ad-hoc solution was used until now. The new code generates compact json that is equivalent to the previous output. I've verified this using jq. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Since xapi 1.249, changes have been attached to xapi versions instead of just a xenserver release. Surface this information to the data that gets used to populate the public documentation webpage Currently it uses a locked API, which might not be maintained in the future, and release dates are not populated which would be good for users to know Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Migration over HTTPS

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Everything except a test already used the "fast path", so the slow version was virtually unused (and slow). Also removes some other dead code. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

To add some protection against overloading the server, and running out of file descriptors and other resources, the number of concurrent connections is now limited to 800 by default. Connections coming in when the limit has been reached are put on hold, and not accepted until another connection finishes. There are three separate buckets for connections to the main unix socket, the client-cert (unix) socket and the TCP socket. This also add some more debug logging when connections are accepted and disconnected. This is disabled by default along with other "http" logs, but can be enabled in the conf file. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…P headers When a connection to xapi's TCP socket is established, no authentication has taken place until the HTTP request has been received. We need to treat these connections with some more care until then. To protect against unauthenticated clients holding connections open without actually making any calls, a read timeout (10 seconds by default) is introduced for every read from the socket until all headers have been read. An HTTP 408 "request timeout" response is returned and the connection broken if a timeout occurs. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

The select system call is inefficient and makes it impossible for a process to use more than 1024 file descriptors. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

CA-368579: Mitigations against DoS attacks by unauthenticated clients

/opt/xensource/debug/with-vdi is a script that create a shell environment with a VDI attached, typically to execute a shell. It is not called during normal operations but can be used Increase the robustness of the script: * Use bash * Exit on errors by default * Improve string quoting * Move to $() command substitution Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

CA-370947 increase robustness of with-vdi script

Because python 2.7 is being used the call to the standard library doesn't support a timeout parameter. In the future this will be useful to protect the script from hanging indefinitely when trying to attach network volumes Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Currently enabling HA can stall because the GFS2 calls when adding the HA statefile might stall indefinitely when there are connection issues Now xapi waits a minute before failing, signalling the user that something has gone awry Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

So far, we identified a GPU by its deviceId; this turns out to be ambiguous and vgpuConfig.xml may contain multiple GPUs with identical deviceId, causing us to use more vGPU types than we can actually support. This patch introduces a new abstract type "gpu" in the VENDOR signature such that its Nvidia instance can use more information to identify a GPU. When matching a GPU found on the PCI bus with GPUs listed in vgpuConfig.xml we use: deviceId match and subsystemId match or deviceId match and subsystemId in vgpuConfig is 0 subsystemId=0 is used by Nvidia as a wildcard and thus subsystemId=0 is ignored during a match. The matching for other vendors remains unchanged (and is still based on deviceId). Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

These releases happen when an API change has been introduced and gen_lifecycle is executed before tagging. They are in the form of 22.11.0-next. For ordering the ints are used like before, and the string is only used to compare if the integers are equal. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

There were two problems here: the version was not being branded int othe binary, producing lifecycle changes with 0.0.dev. and the parser for the version was not detecting correctly the rpm versions because these changed the dashes for dots. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Browsing the changes in the latest version is usually more useful than the first version Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

…eleased

CA-370575: [XSI-1310] Driver disks / supp packs applied at host

…/CA-370578 CA-370578 use subsystemId in NVidia GPU matching

User were allowed to define more than one certificate-chain when installing a host certificate. The implementation picked the first one, ignoring successive ones. This could lead to a situation where the host served a certificate with certificates missing in their chain of trust, making it unverifiable. Instead detect the situation and block the operation immediately. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: jameshensmancitrix <james.hensman@citrix.com>

…jameshen/xsi-1289 CA-369446

Breaks PS SDK generation as the mentions were referring to C# code Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

…ilod/uuid-powershell Revert `uuidx` rename in `gen_powershell_binding.ml`

Without it packages that don't depend on xapi-idl will fail trying to compile it Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This allows to build and test the packages when pinning them in an opam switch Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

There is a mismatch on how this library is called between the xenctrl library used in opam and the one produced by xen, don't mention it for the time being. the names are xenctrl.xenmmap and xenmmap respectively Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This program is used to sync blobs across hosts in a pool. It should verify certificates, as for other TLS connections within a pool. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

CA-371759: check certificates in xsh

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Using this more compact representation allows to see more clearly what can be the causes for raising an generic import exception while grepping Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

…esting Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Check that basic API calls get forwarded Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

…e socket This is the socket created by xapi-guard itself through Conduit, but Conduit doesn't currently have an API to notify us when the socket got created and listen()-ed on. And we don't want to return too early to our caller (the unit test or xenopsd) since they may get an ECONNREFUSED if they try to connect too early. Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Instead of `foo () >>= fun result -> ...` use `let* result = foo () in ...`. Instead of `foo () >|= fun result -> ...` use `let+ result = foo () in ...`. Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The test is commented out because under certain conditions which couldn't be replicated at the end there are _less_ file descriptors that at the beginning. Signed-off-by: Edwin Török <edvin.torok@citrix.com> Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

…force CP-40490: Require --force parameter to destroy VTPMs

…tests

Signed-off-by: Mark Syms <mark.syms@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

These can be replaced by functions in the standard library Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

In this form is more difficult to allow a variable number of elements then using .. :: .. :: .. so it's easier to spot possible issues around them When a call has default arguments it's not possible to change the form as these are optional arguments and make the list variable in length Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This will allow this code to generate a binding not only for the value with the unmarshalled value, but for the rpc value as well. This is needed to be able to explicitely define the parameter list with the default values embedded into the rbac check Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Explicitly define a list of tuples with names and values so the rbac checks can never fail at runtime on differing lengths for these. Instead now the check will fail at build time. Calls with defaults allowed the values list to be shorter as they never bothered to contain the default values. Now the default values are collected to match all the names of the parameters. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

CP-41033: install XenAPI for Python3

…e-receive

Signed-off-by: Mark Syms <mark.syms@citrix.com>

CP-41033: further updates to XenAPIPlugin for Python3

Change the args for Rbac.check to be a pair of lists to be a list of pairs. This removes quite a few error conditions, simplifies the code and reduces the CPU needs of this code. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

It's the same call as get_rrd_updates, which is ignored Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

xcp-rrdd spends 17% of the time in this function, comparing strings using polymorphic compare. Convert current_dss to a set so querying for membership isn't as costly Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Remove unneeded functions, simplify sexpr serialization Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

No need to check whether a key exists and then fetch it, just fetch it Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Instead use specialized comparison functions which have less overhead Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Previous code tried to avoid creating memory garbage by trying comparing the old value of a map with the new when updating it. This unfortunately was done using structural equality instead of physical (its memory location). This dominated the runtime of xapi under some conditions. Instead of trying to optimize use directly the standard library which does the right thing (physical equality), this reduces CPU cost by half in systems with thousands of VMs Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The flag was only turning off the debug message but not the code itself. Indentation now makes it very clear that this is wrong, so lets fix it. Currently this workaround (from the 6.x era is still on in the .conf file, but at some point we may want to turn it off) Signed-off-by: Edwin Török <edvin.torok@citrix.com>

This edits the project file so that a NuGet package is generated at build time. To achieve this this also: - Adds an icon for the NuGet package - Adds a stripped down version of the `README` (`README-NuGet`) that is shown when the package is viewed through a NuGet package manager. The `README` outputs as a Markdown file because it is a requirement. NuGet.org-like managers are smart enough to read the current format properly, without actually using Markdown syntax. - Modify `windows-line-endsings.sh` so that it doesn't corrupt the package icon. - Updated references of Citrix Hypervisor in the README back to XenServer. Citrix references are still present. In fact, they're also present in other SDKs (see Java for instance) Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

Project wouldn't build otherwise Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

…ix-flag

… swtpm-wrapper. Signed-off-by: Jennifer Herbert <jennifer.herbert@citrix.com>

CA-352073, CA-371780, CA-372128: reduce overhead in xapi database and rbac checks; and in xcp-rrdd

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Currently, all callers of sparse_dd still set verfification off, so there is no change in behaviour yet. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Add certificate checking to sparse-dd

We want to tell gpumon to attach and detach its NVML library. Extend the RPC interface defined for gpumon with new calls. Futhermore, generate the gpumon-cli to exercise these operations from the command line. This requires a change in xapi.spec: %files -n message-switch %{_unitdir}/message-switch.service %{_sbindir}/message-switch %{_sbindir}/message-cli /usr/bin/gpumon-cli %config(noreplace) /etc/message-switch.conf /etc/xensource/bugtool/message-switch/stuff.xml /etc/xensource/bugtool/message-switch.xml Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

Provides a far readable output than ounit The strangest thing going around is the change from strings to datasources, the equality comparison still happens only with their names as using Stdlib.(=) fails Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This function had quadratic costs with all the lists it was using. On top of that all three types of owned datasources were processed in a different way, especially the VMs, which where map from a list of active VMs instead of the owned data sources Further work is needed move collect the datasources into the appropriate datastructure instead of in this function, and extend the datastructures into the used functions isntead of converting them to lists Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The change to the structure of the Vdi module allows for the vm associated with a dp on a vdi to be recorded. In the calls DP.destroy and VDI.destroy, a blank quote was given as no vm was available, this has now been remedied by recording and querying this record so there is a valid vm when calling SMAPIv3 interfaces. Signed-off-by: jameshensmancitrix <james.hensman@citrix.com>

Signed-off-by: jameshensmancitrix <james.hensman@citrix.com>

…jameshen/vdi-destory CP-40823 Corrected blank vm in DP.destroy and VDI.destroy functions

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Currently, certificate verification is hardcoded off inside this helper function, which is used mostly in storage migration. So far, this parameter has been effectively unused, since the relevant code path use plain HTTP connections. Now that we will start using HTTPS everywhere, verification needs to be controlled and explicitly configured higher up. The value of verify_cert has not yet been modified, so there is no functional change yet. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

The code that constructs rpc functions has been moved to Storage_utils and extended with options to turn on certificate checking of the remote server. The following commits will set those parameters to the right values depending on whether the storage migration takes place within a pool, or cross pool. Also removed the use of the term "local" around this code, as in some cases these weren't actually used for host-local connections. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…A.copy(_into) This option controls whether or not to verify the certificate of the remote server. This is enabled only for calls (as part of storage migration) within a pool at the moment. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

When starting a storage copy or mirror operation, the call now comes with a verify_dest parameter, indicating whether the remote server's certificate must be verified when connecting to it. These calls return an ID for the operation, which is performed asynchronously. Any following calls regarding the same operation, e.g. to stop or cancel, only take this ID as an argument, but may require new connections to the remote. For this reason, a state record with connection details is maintained for every operation. This commit adds the verify_dest parameter to this state, and to all the places in calls that use the state to make connections. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Similar to DATA.MIRROR.start and DATA.copy(_into). Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

This is used to export VM metadata to another host during live migration. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Jennifer Herbert <jennifer.herbert@citrix.com>

...to decide whether to use certificate checking or not. This module uses a local ref to determine whether checking is enabled on the host. In xapi, this ref is set from a file on startup, and after that it is updated if checking is enabled/disabled through the API. Xenopsd currently does not initialise the ref, so checking is always off. We could initialise the ref on startup, like in xapi, but then xenopsd still won't get updates if the host-level switch is flipped. Instead, let xapi worry about the host-level state, by querying Stunnel_client, and tell xenopsd exactly what to do. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…an values We use `verify_dest` to simply indicate whether to verify certicates or not. The name `verify_cert` is used for the more complex type that includes the CA path and SNI, as defined in the Stunnel module. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

CP-41028: Enforce certificate checking for intra-pool storage migration connections

Signed-off-by: jameshensmancitrix <james.hensman@citrix.com>

This logs the exception and backtrace if the program exits with an uncaught exception. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…jameshen/vdi-destory

There may be a problem with some VMs, but we must always let xenopsd start. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

CA-369444: Ensure xenopsd still starts if VM state upgrade fails

…/cp-40190

By default, logs from the http server are supressed, because it would log far too much. However, it would be valuable to always log the exception that has led to a 500 response, because this error condition is often due to a bug. Logging for these cases is done under a separate name, which is enabled by default. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…/CP-33044 CP-33044 define attach/detach IDL calls for gpumon

CA-371419: Always log exceptions when responding with 500 Internal Error

By default xcp-rrdd does not log debug lines, elevate two messages so it's easier to know when this operation starts happening and when it fails. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Two loglines per call removed: one is not really useful since the first logline already contains the most important information about the stunnel configuration and the one about the cache is not important as the code is known to work as intended. The main logline about stunnel has been changed to make it obvious there has been a change in the area about logging when triaging with logs Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The module ThreadLocalTable uses a Map data structure to keep track of the name to use for a thread in the logs. A name is normally registered through a call to `Debug.with_thread_associated`, soon after a thread is created. However, each time a new context is created through a function like `Server_helpers.exec_with_new_task`, `Debug.with_thread_associated` is called as well, which then overwrites the name for the current thread. When the inner task exits, the old name is not restore, but simply left blank. The result is the useful information end up missing from the first part of log lines. For example. we see this [error||128 ||xenops] rather than [error||128 |xapi events D:4f8a44d7e8d1|xenops] In the past, there was a Hashtbl instead of a Map here, which does remember all previous bindings in the table. The Hashtbl was removed in order to avoid taking the lock in `find`. This patchs adds a list inside each Map value to model a stack, which behaves like Hashtbl, but in a pure functional way. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

The session.logout was used to interrupt the call to `event.from`, so that is can be called again with a new list of VMs to watch, in case a VM started, stopped etc. The trouble is, that each such re-register event led to SESSION_INVALID being logged, with stack trace, three times. This caused many people looking at the logs unnecessary worries, while in reality the behaviour is completely normal. This replaces the register mechanism by adding the event loop's task into the watch list for `event.from`, and injecting an event on the task to interrupt the call and recompute the VM list. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Jennifer Herbert <jennifer.herbert@citrix.com>

Eliminate unnecessarily scary log lines

This is generating a lot of spam which is interfering with testing. blocking releases. The root cause of duplication will be targeted in another PR Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This is currently set on the destination, in host.migrate_receive. However, all recent changes to make HTTPS migration work were on the source host. Migration is allowed from older to newer software versions, so it is possible that the destination is HTTPS capable, while the source is not. This may result in the source receiving HTTPS URLs, which is cannot handle, breaking this upgrade case. Instead, let the source decide whether to switch to HTTPS or not, depending on the config key (to be made the default later). Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Enable HTTPS for storage migration on the source host

This allows the MSBuild CLI to specify assembly version at build time Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

…/cp-41213

The vbd-plug call in with-vdi may fail after a VBD was already created. The errexit option would cause the script to fail before the VBD would be destroyed. So make sure we destroy the VBD if vbd-plug fails. Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

…/CA-372785 CA-372785 make with-vdi more robust

`ModuleToProcess` has been deprecated Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

Xenopsd's _H() in xenctrlext_stubs blindly reaches sideways into the internals of a different libray (specifically, Xenctrl from upstream Xen). This violates many API/ABI rules. It also breaks now that upstream Xen has switched Xenctrl to using a Custom block. Have Xenctrlext open its own libxc handle, to disentangle it from Xenctrl. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Modify Xenctrlext to use its own libxc handle

Also fix some whitespace Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

Signed-off-by: Danilo Del Busso <Danilo.Del.Busso@citrix.com>

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>

- Add `location` to `Http.Request.Hdr` - Add `response_redirect` method to `http-svr` - When `GET` is received by the fileserver and `website-https-only` is true: - if `host` is filled in the request, redirect towards the same URI but in HTTPS - if `host` is not present, reply with a forbidden as before Solves xapi-project#4856 Signed-off-by: BenjiReis <benjamin.reis@vates.fr>

Signed-off-by: Steven Woods <steven.woods@citrix.com>

…https

…-371790

…a/CP-41348

This got removed from Xen 4.18. We only use a global, so switch to using Xenctrl.with_intf. (We can't use a single global 'xc' value, because that might cause some unit tests to fail since initializing xenctrl outside of a Xen system is not possible, so delaying initialization is still needed). Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Xenctrl: drop interface_close

Signed-off-by: Steven Woods <steven.woods@citrix.com>

…-41279 CP-41279: add migration_compression pool option

…ilod/sdk CP-40404: Add C# NuGet specs in its csproj and Move C# SDK samples to .NET 6.0

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

A recent change registered for events on the task that was used by the events_from_xapi thread in xapi_xenops. This is so that the event loop can be woken up by injecting an event into this task, without the need to logout its session (as previously done). The problem is that this introduced a task in the DB that is forever "pending", which has unwanted side-effects. For example, it gets GC'ed after 24 hours, and also gets destroyed if the coordinator is restarted. Switch to registering the host object for the localhost instead, which should be more stable. Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

CA-373551: register for host events rather than task in events_from_xapi

This allows to enforce lifecycles to all fields and messages and xapi to know it, especially when loading the database from a file. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

These we all removed without being deprecated. These would be all rejected by the lifecycle automaton. Only with the SSL legacy it's clear at which released the message got deprecated (as soon as it was introduced) For the rest I used an easily-recognisable message that happens before and at the same release as the removal. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This will block any unsafe transitions from happening in the API Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This allows the database library to be able to depend on the datamodel one, while the latter now depends only the schema library The database library needs to depend on the datamodel one to be able to know when a field is a prototype one Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

No functional changes, the changes try to reduce the complexity of reading the code and reuse code for brevity Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The comment is misleading since it refers to behaviour that has long been forgotten Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This allows dropping values that were only prototyped, or deprecated from the database Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This is the default value and serves no purpose Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This enables build the state at compile time without ad-hoc processing as this is done in the datamodel itself. For example to block removed fields from database constructors when autogenerating their code. This information is encapsulated in Lifecycle.t which is a private record. This makes it easy to deconstruct while forces its construction using the function Lifecycle.from. This prevents inconsistent Lifecycles from being created. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Now the database schema and most of the code generation are filtering these. In the case where the documentation about removal is still needed, the old method is used, albeit with a new name for the avoidance of doubt. This change means that it's impossible to create objects with removed fields, introducing quite a bit of churn. The most salient issues here are: the host metrics drop the free memory, which was still being maintained and used for estimating evacuation time. This is now done using total memory consumed by VMs instead; the removal of metrics in VIFs and VBDs, which makes it impossible to delete these objects from the database; and the removal of vmpp-related fields. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

This change affects the public documentation only. When a field is published and deprecated on the same release, only the the deprecation notice is shown. Similarly when a message is deprecated and removed in the same release, only the removal gets shown. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>

…in-the-rain

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>

Other distributions other than Xenserver are used for testing the installation of packages, and they have removed python as an alias for python2. Being explicit doesn't break compatibility on Xenserver, and allows us to keep newer versions of Ubuntu for testing in github Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>

The former is technical debt in the hypervisor patchqueue, while the latter is an upstream binding. No functional change. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Switch from Xenctrl.hvm_check_pvdriver to Xenctrl.hvm_param_get

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Now keys using elliptical curve cryptography need to be taken into account. Keep the same behaviour as previously by rejecting them Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

The context is now is lazy-loaded, so force it's loading. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Now io-page is compiled using dune and it's not split into two. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Makes it easier to spot which call failed this check when triaging related issues Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

If a file descriptor is open in the normal way then its flags will be 0, and the code here that attempts to enable O_DIRECT will do nothing. This appears to have been an incorrect way of inverting an error condition `ret < 0`, the negation of which is `ret >= 0`, and not `ret > 0`. We open the file with O_DIRECT elsewhere though (in vhd-format), but fix this code to avoid confusion. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

compile_flags.txt (or compile_commands.json) can be used by C language servers, like 'clangd' to provide in-editor feedback about C code compile warnings/errors (similar to ocaml-lsp). By default it wouldn't know where to find the OCaml includes and show a lot of warnings/errors when editing C stubs, and with this file it works. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

There is no need to declare it by hand (it also has a slightly different prototype taking a 'const char*' as argument). Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Signed-off-by: Edwin Török <edvin.torok@citrix.com>

…rno which is It is usually set to XC_INTERNAL_ERROR, except in xenguest, but we have no bindings for that. This API might get deleted in upstream Xen too: the error code is stored in the xenctrl handle, but we made that global per process, and shared among all threads. Which means that xenctrl calls from different threads can overwrite each-others' error code. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

This is a C call, and might benefit from a bit more parallelism if it inside the blocking section. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Most of these calls pass only integers or pointers allocated in C, so it is safe to release the OCaml runtime lock. Replace direct passing of Xfm_val/Caml_ba_data_val with storing in a temporary local variable (the result of Xfm_val or Caml_ba_data_val won't move since they are C pointers, but the OCaml value passed in as params to these macros might!) Signed-off-by: Edwin Török <edvin.torok@citrix.com>

…section where possible unshadow is not thread safe, but the other ones should be. TODO: double check all C API calls in unixpwd.c with the MT-safe portion of the manpage Signed-off-by: Edwin Török <edvin.torok@citrix.com>

We may intend to upstream the xenctrlext stubs to Xen, so it should follow the Xen CODING_STYLE. the .clang-format here is based on the version I sent upstream that tries to follow that coding style. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Xenmmap.read is being removed from Xen, use upstream mirage libs to access this instead. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

XXX: Nested virt now broken Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Starting all services at once is what happens on boot, so this should work. We currently stop services one at a time, and if they get stuck (e.g. due to waiting on a network filesystem that is stuck) then each has to hit the 1m30s timeout in systemd before they get killed, which is a minimum of ~32m before the toolstack is stopped. Which is a lot higher than the default 2 minute HA timeout, and even if the XAPI watcher would attempt recovery by restarting XAPI (or the toolstack), it would be stuck waiting for this 30m process to complete first. Stop all services at once and then start all needed ones at once. (Don't attempt to restart all in a single command since that might result in some services restarting and immediately seeing another service being restarted, interrupting communication and potentially leading to errors). Systemd can then use existing service dependencies to order the restarts. Signed-off-by: Edwin Török <edvin.torok@citrix.com>

Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>

Signed-off-by: Edwin Török <edwin.torok@cloud.com>

… due to ifdefs

…in CI

Private/edvint/static #1

Private/edvint/static #1

Commits on Oct 4, 2022

Commits on Oct 5, 2022

Commits on Oct 6, 2022

Commits on Oct 7, 2022

Commits on Oct 10, 2022

Commits on Oct 11, 2022

Commits on Oct 12, 2022

Commits on Oct 13, 2022

Commits on Oct 17, 2022

Commits on Oct 18, 2022

Commits on Oct 19, 2022

Commits on Oct 20, 2022

Commits on Oct 21, 2022

Commits on Oct 24, 2022

Commits on Oct 25, 2022

Commits on Oct 26, 2022

Commits on Oct 27, 2022

Commits on Oct 28, 2022

Commits on Oct 31, 2022

Commits on Nov 2, 2022

Commits on Nov 3, 2022

Commits on Nov 7, 2022

Commits on Nov 8, 2022

Commits on Nov 14, 2022

Commits on Nov 15, 2022

Commits on Nov 16, 2022

Commits on Nov 17, 2022

Commits on Nov 18, 2022

Commits on Nov 21, 2022

Commits on Nov 23, 2022

Commits on Nov 24, 2022

Commits on Nov 25, 2022

Commits on Nov 28, 2022

Commits on Nov 29, 2022

Commits on Nov 30, 2022

Commits on Dec 1, 2022

Commits on Dec 5, 2022

Commits on Dec 6, 2022

Commits on Dec 7, 2022

Commits on Dec 9, 2022

Commits on Dec 12, 2022

Commits on Dec 14, 2022

Commits on Dec 15, 2022

Commits on Dec 16, 2022

Commits on Dec 21, 2022

Commits on Jan 4, 2023

Commits on Jan 5, 2023

Commits on Jan 10, 2023