Porting lwIP for an OS

The operating system emulation layer is located in two files, cc.h and sys_arch.c. It provides a common interface between the lwIP code and the underlying operating system kernel. The general idea is that porting lwIP to new architectures requires only small changes to a few header files and a new sys_arch implementation. It is also possible to do a sys_arch implementation that does not rely on any underlying operating system.

cc.h

This is a basic header that describes your compiler and processor to lwIP. The following things must be defined:

sys_arch.c

The sys_arch provides semaphores and mailboxes to lwIP. For the full lwIP functionality, multiple threads support can be implemented in the sys_arch, but this is not required for the basic lwIP functionality. Previous versions of lwIP required the sys_arch to implement timer scheduling as well but as of lwIP 0.5 this is implemented in a higher layer.

Semaphores

Semaphores can be either counting or binary - lwIP works with both kinds. Semaphores are represented by the type sys_sem_t which is typedef'd in the sys_arch.h file. lwIP does not place any restrictions on how sys_sem_t should be defined or represented internally, but typically it is a pointer to an operating system semaphore or a struct wrapper for an operating system semaphore.

The following functions must be defined:

Note that there is another function sys_sem_wait in sys.c, but it is a wrapper for the sys_arch_sem_wait function. Please note that it is important for the semaphores to return an accurate count of elapsed milliseconds, since they are used to schedule timers in lwIP. See the timer section below for more information.

Mailboxes

Mailboxes are used for message passing and can be implemented either as a queue which allows multiple messages to be posted to a mailbox, or as a rendez-vous point where only one message can be posted at a time. lwIP works with both kinds, but the former type will be more efficient. A message in a mailbox is just a pointer, nothing more.

Mailboxes are equivalently represented by the type sys_mbox_t. lwIP does not place any restrictions on how sys_mbox_t is represented internally, but it is typically a structure pointer type to a structure that wraps the OS-native mailbox type and its queue buffer.

The following functions must be defined:

lwIP timeouts

Many protocols require certain events to happen after certain time periods. For example, if an ACK is not received within a certain time window in TCP, the packet needs to be retransmitted. For simplicity in implementation, lwIP piggybacks the timer requirement onto the mailbox and semaphores.

Although complete understanding of timeouts is not required to properly implement sys_arch.c, it does help explain what the one function you must provide actually does.

QUICK EXPLANATION: The linked list of timeouts is a linked list of struct sys_timeo's. This structure holds the number of milliseconds left until it should be activated, as well as a pointer to the function and its argument that should be called on timeout. When a thread (or the program itself) blocks on a lwIP semaphore (via sys_sem_wait) or on a mailbox (via sys_mailbox_fetch), it blocks for a time equal to the next timeout that needs servicing. If the semaphore or mailbox times out, the timeout handlers needing serviced are called. Otherwise, all the timeouts are decremented by the amount of time elapsed in the blocking call (hence why it is necessary for sys_sem_wait to accurately report how long the call was blocked).

The linked list is contained in a struct sys_timeouts. The function you must implement is:

Again, this is a bit vague. Here is an example implementation:

struct sys_timeouts lwip_system_timeouts = NULL; // Default timeouts list for lwIP

struct sys_timeouts *sys_arch_timeouts(void) {
return &lwip_system_timeouts;
}

If you do not use threads in your system, then this implementation is all you need. If you do use threads, then you will need to give each thread its own sys_timeouts structure, and return that depending on what thread calls this function. This will be described in the next section.

Threads

Threads are not required for lwIP, although lwIP is written to use them efficiently. The following function will be used to instantiate a thread for lwIP:

If you use threads, then it is important that timeouts happen in the right contexts (specifically the context that it was created in). Therefore, the sys_arch_timeouts function should return a timeouts structure depending on what thread is asking for it (this is actually the reason why this function is needed in the first place, since otherwise, sys.c could simply implement the version above and provide its own single global sys_timeouts variable). One possible impelmentation is shown below, somewhat in pseudo-code:

struct thread_struct_wrapper {
struct thread_struct_wrapper *next;
MY_OS_THREAD_TYPE thread; // not a ptr in this example, but the actual space
struct sys_timeo *timeouts;
};

struct sys_timeouts lwip_system_timeouts = NULL; // Default timeouts list for lwIP
struct thread_struct_wrapper *lwip_system_threads = NULL; // a list of all threads created by lwIP

sys_thread_t sys_thread_new(void (* thread)(void *arg), void *arg, int prio) {
sys_thread_t newthread;
SYS_ARCH_DECL_PROTECT(old_val);
newthread = malloc(sizeof(struct thread_struct_wrapper)); // allocate the space for the thread wrapper
if (newthread==NULL) return NULL;
SYS_ARCH_PROTECT(old_val); // Need to protect this -- preemption here could be a problem!
newthread.next = lwip_system_threads;
lwip_system_threads = newthread;
SYS_ARCH_UNPROTECT(old_val);
newthread.timeouts = NULL; // initialize the linked list to NULL
my_system_os_create_thread_function(&newthread.thread, thread, arg, prio); // create it, however my OS does it
return newthread;
}

struct sys_timeouts *sys_arch_timeouts(void) {
sys_thread_t thread = lwip_system_threads;
MY_OS_THREAD_TYPE *self = my_system_os_get_current_thread();
// Search the threads list for the thread that is currently running
for ( ; thread!=NULL; thread=thread->next) {
if (thread->thread == self) return thread->timeouts;
}
// No match, so just return the system-wide default version
return &lwip_system_timeouts;
}

Final declarations for sys_arch.h

Final thoughts

Be careful with using mem_malloc() in sys_arch. Specifically, when mem_init() is run (in mem.c), it allocates a semaphore to block concurrent access in its routines. If sys_sem_new() calls mem_malloc to allocate its semaphore, then it will be calling mem_malloc() before the mem.c system has been initialized!