pike.git / multi-cpu.txt

version» Context lines:

pike.git/multi-cpu.txt:417:    get arbitrarily large.      e. The double-linked lists aren't needed. Hence two pointers less per    thing.      f. The refcounter word is changed to hold both normal refcount, weak    count, and flags. Overflowed counts are stored in a separate hash    table.      g. The collector typically runs concurrently with the rest of the -  program. It sometimes interrupts the other threads for handshakes. -  These interrupts are not aligned with the evaluator callback -  calls, since that would cause too much pausing of the collector -  thread. This requires that threads can be stopped and resumed -  preemptively. See issue "Preemptive thread suspension". +  program, but there are some situations when it has to synchronize +  with them (aka handshake). In the research paper this is done by +  letting the gc thread suspend and resume the other threads (one at +  a time). Since preemptive suspend and resume operations are +  generally unsupported in thread libraries (c.f. issue "Preemptive +  thread suspension"), a cooperative approach is necessary:    -  +  The gc thread sets a state flag that all other threads need a +  handshake. Threads that are running do the handshake work +  themselves before waiting on a mutex or in the next evaluator +  callback call, and the gc thread handles the threads that are +  currently waiting (ensuring that they don't start in the +  meantime). +  +  The work that needs to be done during a handshake is to set some +  flags and record some local thread state for use by the gc thread. +  This can be done concurrently in several threads, so no locking is +  necessary. +  +  Due to this interaction with the other threads, it's vital that +  the gc thread does not hold any mutex, and that it takes care to +  avoid being stopped (e.g. through an interrupt) while it works on +  behalf on another thread. +    h. All garbage collection, both for noncyclic and cyclic garbage, are    discovered and handled by the gc thread. The other threads never    frees any block known to the gc.      i. An effect of the above is that all garbage is discovered by a    separate collector thread which doesn't execute any other pike    code. This opens up the issue on how to call destruct functions.       At least thread local things should reasonably get their destruct    calls in that thread. A problem is however what to do when that
pike.git/multi-cpu.txt:468:       All the hash table lookups would however incur a significant    overhead in the gc thread. A better alternative would be to use a    bitmap based on the possible allocation slots used by the malloc    implementation, but that would require very tight integration with    the malloc system. The bitmap could work with only two bits per    refcounter - research shows that most objects in a refcounted heap    have very few refs. Overflowing (a.k.a. "stuck") refcounters at 3    would then be stored in a hash table.    - m. FIXME: Is the third NOP handshake really necessary? -  +    To simplify memory handling, the gc should be used consistently on all   heap structs, regardless whether they are pike visible things or not.   An interesting question is whether the type info for every struct   (more concretely, the address of some area where the gc can find the   functions it needs to handle the struct) is carried in the struct   itself (through a new pointer field), or if it continues to be carried   in the context for every pointer to the struct (e.g. in the type field   in svalues).      Since the gc would be used for most internal structs as well, which
pike.git/multi-cpu.txt:1258:   Survey of platform support:      o Windows/Visual Studio: Got "Interlocked Variable Access":    http://msdn.microsoft.com/en-us/library/ms684122.aspx      o FIXME: More..         Issue: Preemptive thread suspension    - The proposed gc should preferably be able to suspend other threads - preemptively (see issue "Garbage collector", item g). Survey of - platform support for this: + The proposed gc as presented in the research paper needs to suspend + and resume other threads. A survey of platform support for preemptive + thread suspension:      o POSIX threads: No support. Deprecated and removed from the standard    since it can very easily lead to deadlocks. On some systems there    might still be a pthread_suspend function.      o Windows: SuspendThread and ResumeThread exists but are only    intended for use by debuggers.    - It's clear that a nonpreemptive fallback is required. + It's clear that a nonpreemptive method is required. See issue "Garbage + collector" item g for details on that.    - Regardless of method, it's vital that the gc thread does not hold any - mutex, and that it takes care to avoid being stopped while it suspends - another thread. This is more important if a preemptive method is used. +     -  +    Issue: OpenMP      OpenMP (see www.openmp.org) is a system to parallelize code using   pragmas that are inserted into the code blocks. It can be used to   easily parallelize otherwise serial internal algorithms like searching   and all sorts of loops over arrays etc. Thus it addresses a different   problem than the high-level parallelizing architecture above, but it   might provide significant improvements nevertheless.      It's therefore worthwhile to look into how this can be deployed in the