|
Udipe 1.0
Solving the riddle of high-throughput UDP
|
Asynchronous operation management. More...
#include "context.h"#include "nodiscard.h"#include "pointer.h"#include "result.h"#include "time.h"#include "visibility.h"#include <assert.h>#include <stdbool.h>#include <stddef.h>#include <stdint.h>Go to the source code of this file.
Typedefs | |
| typedef struct udipe_future_s | udipe_future_t |
Asynchronous libudipe commands, whose name starts with udipe_start_, such as udipe_start_connect(), do not wait for the associated operation to complete and return its result. Instead they return a pointer to a udipe_future_t object that can be used to wait for the result to come up among other things.
Adding this intermediary stage where the command has been submitted to worker threads, but has not been awaited yet, provides a lot of flexibility in the submission and scheduling of I/O-related work. See the documentation of udipe_future_t for more information.
| typedef struct udipe_future_s udipe_future_t |
Asynchronous operation future
udipe_future_t is the heart of the asynchronous API of udipe, which itself is the recommended "default" API for situations where you don't precisely know the performance requirements of your application and want to get a good balance between ergonomics, flexibility and performance.
Every libudipe function whose name starts with udipe_start_ is an asynchronous function. It returns to its caller as quickly as possible, usually before the associated operation has completed. As opposed to returning the operation's result, like a synchronous function would, an asynchronous function instead returns a pointer to a future object, which can be used to interact with the associated asynchronous operation in the ways that are described below.
During normal program execution, each of these futures must eventually be passed to udipe_finish(), which is the point at which the operation's result or errors will be reported, and associated resources will be liberated. This operation is blocking by nature, though we will later see that when the situation demands it, it is possible to have extra flexibility in how the associated waiting is carried out.
Once a future has been passed to udipe_finish(), the ressources associated with it should be considered liberated (even though the actual liberation may not occur immediately), and it must not be used again.
As a general rule, multithreaded programs must be careful not to call udipe_finish() at a time where other threads might still be using the future in any manner. This is true of all uses of a future including waiting for its completion with udipe_wait().
In situations where there are multiple asynchronous operations in flight, it is possible to await the associated futures one by one via udipe_finish(). But this future usage pattern comes with two important limitations, which can be adressed by leveraging udipe's collective operations:
Network commands come with a udipe_future_t* after option. By setting this option to point to the future associated with another asynchronous operation, you can schedule the network transaction of interest to only execute after that asynchronous operation has successfully completed.
When combined with udipe_start_join(), this feature lets you depend on as many futures as you want, enabling you to express arbitrarily complex dependency graphs.
Sometimes, this can reduce the need for application threads to constantly block waiting for asynchronous operations. For example, a basic network packet forwarding task can be expressed as a chain of network receive and send commands operating on the same buffer, where the send command is scheduled to execute after the receive command completes, at which point the application thread can just wait on the final receive commands in order to wait for the entire task to complete. And if you have multiple packets to forward, you can just amortize waiting overhead further by chaining a bunch of (recv, send) futures upfront.
But with that being said, users of other future-based asynchronous programming frameworks will quickly notice that the asynchronous chaining capabilities of udipe are less powerful than those found elsewhere, and notably exclude the ability to post-process input data via a user-defined callback, an operation known in computer science circles as "monadic map".
This omission is intentional and stems from the fact that in its asynchronous API, udipe enforces maximal isolation between network threads and application threads. On one side, network threads execute tightly performance-tuned code under a strict soft realtime discipline to minimize UDP packet loss. On the other side, application threads are free to receive as much or little performance tuning as the use case demands. By forcing network threads to execute arbitrary application code at the time where they signal some I/O completion, monadic map pokes a hole in this isolation, and thus has no place in the asynchronous udipe API.
Now, performance-conscious users may object that executing very simple data post-processing callbacks in network threads can have performance benefits with respect to offloading that processing to application threads via a thread synchronization transaction. But those performance benefits are largely voided by the fact that in the asynchronous udipe API, starting any network command usually involves a thread synchronization transaction, as does waiting for said command to finish executing, so this API is not exactly light on synchronization transactions to begin with. It is one area where the asynchronous udipe API trades performance for ergonomics.
In scenarios where performance becomes the top concern and reduced ergonomics is an acceptable price to pay for it, it is instead recommended to switch to the more advanced callback-based API of udipe. By putting you in the driver seat of network threads, this API lets you get to the optimum of zero thread synchronization per UDP packet in basic operation, and is therefore the recommendation for the most performance-demanding applications for which the asynchronous API is not efficient enough. (TODO: point to more resources once callback API is available).
There are several situations in which an asynchronous command that seemed sensible at the time where it was initiated, turns out to be unnecessary as time passes and more information surfaces:
To handle these situations correctly, some sort of cancelation support is needed. Which is why the asynchronous API of udipe comes with the following asynchronous operation cancelation support:
Sometimes, networking commands need to execute not in relation to each other, but in relation to external factors such as the passing of time. Bearing this in mind, the asynchronous udipe API comes with utilities that let you sync up with the system clock in various ways:
As anyone with asynchronous programming experience can attest, async frameworks are all fun and games until the day where you need to compose them with some awaitable event that the asynchronous framework designer did not think about, and then the eternal suffering begins.
In an attempt to at least plan ahead for such use cases, without over-engineering itself into the corner of supporting arbitrarily complex and OS-specific operations beyond its intended scope, udipe is able to interoperate with application-defined events via a purposely minimal API:
By exception to the normal udipe future lifetime rules, custom futures can be acted upon via udipe_custom_ functions after they are passed to udipe_finish(), but before they are passed to either udipe_custom_try_set_result() or udipe_custom_finish_cancel(). However, in the interest of interface consistency with other future types, it remains an error to pass them to any other future-based function after they have been passed to udipe_finish().
While custom futures may seem convenient on paper, prospective users should be warned that their API is heavily constrained by limitations of the C programming languages and udipe internals. Custom futures therefore trade increased flexibility for poor type safety, an inconvenient API, and deadlock hazards, which is why they should only be used sparingly and in situations where no native udipe abstraction fits.
If you find yourself reaching for custom futures often, the recommended course of action is to contact the udipe developers so that we can figure out together how to provide better support for your use case.
| UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_PUBLIC bool udipe_cancel | ( | udipe_future_t * | future, |
| bool | finish | ||
| ) |
Cancel an asynchronous operation
This function notifies the udipe infrastructure that the asynchronous operation associated with a certain future is not desired anymore and should terminate as quickly as possible, avoiding any work that had not begun yet including dependent work scheduled after the target future.
Like any other asynchronous cancelation APIs, this operation is inherently racy and therefore comes with a lot of caveats that must be kept in mind while using it:
In other words, though latency and UX considerations will sometimes dictate otherwise, the most resource-efficient way to cancel some work remains obviously to not start the work until the point where you know for sure you are going to need it.
If the finish flag is set, then after canceling the operation, this function additionally waits for the operation to terminate then liberates associated resources as if udipe_finish() were called. If finish is not set, then udipe_cancel() returns immediately and must be followed by an udipe_finish() call (possibly preceded by some udipe_wait() if bounded-duration waits are desired) to finish resource cleanup.
It must be understood that the finish flag is only safe to set in situations where you know that no other thread is waiting or could start waiting for the future. A typical use case for it is single-threaded workflows when you have started directly or indirectly waiting for a future, then realize you don't need to wait for this particular operation after all.
| future | must be a future that was returned by an asynchronous function (those whose name begins with udipe_start_) and has not been liberated by udipe_finish() or udipe_cancel() since. If finish is set, then the future will be destroyed by this function and must not be used afterwards. |
| finish | indicates whether udipe_cancel() should wait for the operation to terminate and clean up associated resources, as if udipe_finish() was automatically called afterwards, or solely send a cancelation notification and leave waiting and cleanup to a later manual call to udipe_finish(). |
| UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_PUBLIC bool udipe_custom_cancelled | ( | udipe_future_t * | custom | ) |
Check if a custom future was canceled via udipe_cancel()
Custom tasks that support being interrupted should periodically check this function. If it starts returning true, they should stop doing what they are doing as early as possible, then call udipe_custom_finish_cancel() to acknowledge the cancelation signal.
Custom tasks that do not support being interrupted do not need to bother with this periodical checking and can simply run to completion then call udipe_custom_try_set_result() at the end. The call will fail without changing the future result, which is how they will know that a cancelation signal has been received.
| custom | must be a custom future that was created with udipe_start_custom() and hasn't been passed to udipe_custom_finish_cancel() or udipe_custom_try_set_result() yet. By exception to the normal udipe future lifetime rules, it is valid to pass in a future that has already been passed to udipe_finish(). |
| UDIPE_NON_NULL_ARGS UDIPE_PUBLIC void udipe_custom_finish_cancel | ( | udipe_future_t * | custom | ) |
Acknowledge the cancelation of a custom future
After receiving the udipe_custom_cancelled() signal, a custom task should interrupt its work as quickly as possible, then call this function to acknowledge that it is done canceling itself and will not perform any further processing related to its initially scheduled task.
This will have the effect of waking up downstream threads that were waiting for this task to complete, with a notification that it was canceled. At this point, they should be free of modifying or reallocating any input data pointer that was passed in at the time where the custom task was created, without any risk of racing with the thread that implements the custom task. In particular,
It is an error to call this function on a future that was not passed to udipe_cancel() and therefore never reported being canceled via udipe_custom_canceled().
| custom | must be a custom future that was created with udipe_start_custom() and cancelled via udipe_cancel(), but hasn't been passed to udipe_custom_finish_cancel() or udipe_custom_try_set_result() yet. By exception to the normal udipe future lifetime rules, it is valid to pass in a future that has already been passed to udipe_finish(). But after being passed to this function, the future must never be passed to any other udipe_custom_ function again. |
| UDIPE_NON_NULL_ARGS UDIPE_PUBLIC bool udipe_custom_try_set_result | ( | udipe_future_t * | custom, |
| bool | successful, | ||
| udipe_custom_payload_t | payload | ||
| ) |
Attempt to set the end result of a custom operation
This will normally mark the future as completed with the specified result, unless it was previously canceled, in which case the operation will fail and report this failure by returning false.
When this happens, the future will still be marked as completed, but without a result. Instead it will be in a canceled state.
| custom | must be a custom future that was created with udipe_start_custom() and hasn't been passed to udipe_custom_finish_cancel() or udipe_custom_try_set_result() yet. By exception to the normal udipe future lifetime rules, it is valid to pass in a future that has already been passed to udipe_finish(). But after being passed to this function, the future must never be passed to any other udipe_custom_ function again. |
| successful | indicates whether the custom operation should be considered successful, in the sense that other futures scheduled after this one should be allowed to start executing. |
| payload | is a custom data block of your choosing, which encodes the end result of the custom operation. In case of success, it can be used to pass down the result of the operation if it was not transmitted by other means like filling a user-provided buffer. In case of failure, it should encode any information to take appropriate error handling action needed downstream. |
false will be returned, otherwise true will be returned. | UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_PUBLIC udipe_result_t udipe_finish | ( | udipe_future_t * | future | ) |
Wait for the end of an asynchronous operation, collect its result and schedule the liberation of associated resources
From the point where any thread enters this function, future should be treated as liberated. In other words, udipe_finish() cannot be called until all previous calls to the udipe API that involve this future have returned, and this future cannot be passed to any other udipe function by any thread anymore after the point where udipe_finish() has started being called.
One consequence of this rule is that a future which has been passed to udipe_finish() cannot be passed to udipe_cancel(), and thus the implicit wait of udipe_finish() cannot be canceled. To achieve cancelable wait or any kind of concurrent waiting by multiple threads, you will need to use udipe_wait() first instead of calling udipe_finish() directly.
By the time this function returns, it is guaranteed that...
| future | must be a future that was returned by an asynchronous function (those whose name begins with udipe_start_) and has not been liberated by udipe_finish() or udipe_cancel() since. It will be destroyed by this function and must not be used afterwards. |
| UDIPE_PUBLIC UDIPE_NON_NULL_ARGS void udipe_join | ( | udipe_context_t * | context, |
| udipe_future_t *const | futures[], | ||
| size_t | num_futures | ||
| ) |
Eagerly wait for multiple asynchronous operations to all terminate
This is the synchronous version of udipe_start_join(), which can be used when you want to wait for multiple asynchronous operations to finish and have nothing else to do meanwhile.
On some platforms, the implementation of udipe_join() may be more efficient than that of simply calling udipe_wait() sequentially for each input future. But bear in mind that you WILL need to call udipe_finish() on each of the input futures eventually in order to check out their results and liberate associated state. All udipe_join() guarantees is that these calls will be nonblocking, which should reduce their individual overhead.
| context | must point to an udipe_context_t that has been set up via udipe_initialize() and hasn't been liberated via udipe_finalize() since. |
| futures | must point to an array of at least one futures that were all returned by an asynchronous function (those whose name begins with udipe_start_) and have not been liberated by udipe_finish() or udipe_cancel() since. |
| num_futures | must match the length of the futures array, and thus be at least one. |
| UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_NON_NULL_RESULT UDIPE_PUBLIC udipe_future_t * udipe_start_custom | ( | udipe_context_t * | context | ) |
Create a custom future that will complete with a result of your choosing once udipe_set_custom() is called on it.
Custom futures enable the asynchronous udipe API to interoperate with other asynchronous and blocking system APIs, at a price: due to design constraints from both udipe and the C type system, the resulting API is rather error-prone. Therefore, if you find yourself using custom futures often, you should consider contacting the udipe development team to see if your use case could gain first-class support in udipe.
Custom futures can be passed to all usual future-based APIs (udipe_finish(), udipe_wait(), udipe_join()...), but in addition they support two extra operations:
By nature, custom futures introduce deadlock hazards. For example, this code will instantly deadlock for hopefully obvious reasons:
One less obvious avenue for deadlock, however, is network thread backpressure. To prevent a client submitting work faster than it can be processed from trashing CPU caches and eventually exhausting system RAM, network command submission becomes blocking once the number of waiting tasks reaches a certain threshold. One side-effect of this safety feature is that the following custom future usage pattern is also unsafe:
The aforementioned deadlock hazards can often be avoided by following some relatively simple deadlock safety principles…
In practice, these principles can often be honored by segregating your application threads into "udipe threads" on one side that schedule and await udipe work, and "non-udipe threads" on the other side that eagerly perform work then signal its completion to the udipe threads, without ever using any udipe functionality other than creating custom futures, checking for cancelation and submitting results in the process.
| UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_NON_NULL_RESULT UDIPE_PUBLIC udipe_future_t * udipe_start_join | ( | udipe_context_t * | context, |
| udipe_future_t *const | futures[], | ||
| size_t | num_futures | ||
| ) |
Start waiting for multiple asynchronous operations to terminate.
This function returns a future that will complete with an empty result once all upstream operations have completed, or become canceled if at least one upstream operation is canceled or errors out.
This operation serves several purposes:
For related use cases, you may also consider using...
| context | must point to an udipe_context_t that has been set up via udipe_initialize() and hasn't been liberated via udipe_finalize() since. |
| futures | must point to an array of at least one futures that were all returned by an asynchronous function (those whose name begins with udipe_start_) and have not been liberated by udipe_finish() or udipe_cancel() since. The output future will retain a pointer to this array, which must therefore not be modified or liberated until the completion of the output future has been awaited via udipe_finish() or udipe_cancel(). |
| num_futures | must match the length of the futures array, and thus be at least one. |
futures can then be fetched in a non-blocking manner by simply calling udipe_finish() on each of them in a sequence. | UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_NON_NULL_RESULT UDIPE_PUBLIC udipe_future_t * udipe_start_timer_once | ( | udipe_context_t * | context, |
| const struct timespec * | ts | ||
| ) |
Return a one-shot timer future that will complete with no result once a specific absolute time is reached
The target time is specified in the same format that is output by the C11 timespec_get() function in TIME_UTC mode. On both Unices and Windows, if the system clock is set up correctly, this will corresponds to a number of seconds and nanoseconds elapsed since the Unix epoch (Midnight, January 1, 1970, UTC), with only approximate handling of leap seconds. This behavior matches that of the CLOCK_REALTIME system clock defined by POSIX's clock_gettime().
This future is not normally used in isolation. It is rather chained before other futures to schedule them for execution at a specific time, or combined with other futures through udipe_start_unordered() when you need an absolute timeout rather than a relative one. In this latter role, one notable property of timer futures is that they can have finer time resolution than standard operating system timeouts on udipe_wait(), at the expense of being more expensive to set up.
| context | must point to an udipe_context_t that has been set up via udipe_initialize() and hasn't been liberated via udipe_finalize() since. |
| ts | must point to a valid struct timespec indicating at which time the wait will complete, following the conventions outlined above. |
| UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_NON_NULL_RESULT UDIPE_PUBLIC udipe_future_t * udipe_start_timer_repeat | ( | udipe_context_t * | context, |
| const struct timespec * | initial, | ||
| udipe_duration_ns_t | interval | ||
| ) |
Return a repeating timer future that will first complete once a specific absolute time is reached, then yield a chain of other futures that complete following subsequent timer ticks at a specified time interval.
At udipe_finish() time, each future within the chain will also tell you how many of the specified timer ticks were missed, which can be used to detect situations where your application is not keeping up with user-specified periodicity constraints and should take corrective actions to get back in the desired performance range.
For example, let's say that you specify an initial time 3s into the future (we'll call that T+3s), then an interval of 100ms. This results in a timer with ticks at T+3s, T+3.1s, T+3.2s, etc.
| context | must point to an udipe_context_t that has been set up via udipe_initialize() and hasn't been liberated via udipe_finalize() since. |
| initial | must point to a valid struct timespec indicating at which time the first yielded future will complete, following the conventions outlined in the documentation of udipe_start_timer_once(). |
| interval | must specify a nonzero number of nanoseconds to await between between timer ticks, which subsequent futures in the chain will report. There is no UDIPE_DURATION_DEFAULT for this function. |
initial time point has been reached, yielding a number of missed timer ticks and a chain of other futures that complete following a regular cadence given by interval. | UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_NON_NULL_RESULT UDIPE_PUBLIC udipe_future_t * udipe_start_unordered | ( | udipe_context_t * | context, |
| udipe_future_t *const | futures[], | ||
| size_t | num_futures | ||
| ) |
Start waiting for multiple asynchronous operations to terminate, returning a chain of futures that will terminate as the upstream operations terminate
This asynchronous command is somewhat similar to udipe_start_join(), in that it produces a future that lets you asynchronously wait for multiple futures to terminate whenever you are ready for it. However, in contrast with udipe_start_join(), the wait is more fine-grained.
The future that you get out of of udipe_start_unordered() lets you wait for the first upstream operation to complete, then tells you which operation completed and gives you another future that lets you wait for the second upstream operation to complete, and so on until all initially specified operations have completed.
Compared to joined futures, unordered futures have some benefits...
...but unordered execution also has some drawbacks that must be kept in mind before blindly using it all the time for all purposes.
futures array), and thus will cost more CPU time on the client thread than joined futures....which is why you should not be afraid of mixing and matching udipe_start_join() with udipe_start_unordered() as appropriate, maybe even using both at the same time on the same set of futures in complex use cases.
| context | must point to an udipe_context_t that has been set up via udipe_initialize() and hasn't been liberated via udipe_finalize() since. |
| futures | must point to an array of at least one futures that were all returned by an asynchronous function (those whose name begins with udipe_start_) and have not been liberated by udipe_finish() or udipe_cancel() since. The output futures will retain a pointer to this array, which must therefore not be modified or liberated until the completion of all output futures has been awaited via udipe_finish() or has been canceled via udipe_cancel(). |
| num_futures | must match the length of the futures array, and thus be at least one. |
| UDIPE_NODISCARD UDIPE_NON_NULL_ARGS UDIPE_PUBLIC bool udipe_wait | ( | udipe_future_t * | future, |
| udipe_duration_ns_t | timeout | ||
| ) |
Wait up to a certain duration for the end of an asynchronous operation
In contrast with udipe_finish(), which also waits for asynchronous operations to terminate, this function only waits for a result to be available, it does not fetch that result and liberates resources. In other words, udipe_wait()...
timeout plus some extra delay. This extra delay is normally a small processing overhead in the microsecond range, but sadly some underlying OS APIs only support timeouts with millisecond granularity, which increases the minimal timeout to 1ms.Here are some examples of use cases that you can handle by calling udipe_wait() first instead of calling udipe_finish() directly:
udipe_wait(), but bear in mind that you will need to be extra careful as you still need to call udipe_finish() on one thread eventually and it can only be done after all other threads have returned from their wait. For this use case, it is normally better to have one thread that is responsible for awaiting, fetching and broadcasting the result, which other threads will await via a broadcast synchronization primitive under your control such as a condition variable.| future | must be a future that was returned by an asynchronous function (those whose name begins with udipe_start_) and has not been liberated by udipe_finish() or udipe_cancel() since. |
| timeout | must be a timeout in nanoseconds, after which this function will give up and return false if the asynchronous operation has not completed yet. Special value UDIPE_DURATION_MIN can be used if you just want to non-blockingly check if the operation has completed. |
future has completed and its result is ready to be fetched. If this function returns true (which is guaranteed when timeout is UDIPE_DURATION_MAX), calling udipe_finish() on the same future is guaranteed to return a result immediately without blocking. 1.9.8