Linux Time Management

Today's Lecture

• Q&A
• Kernel Programming Hints
• When Will Tasklet Run?
• Kernel Synchronization

Kernel Programming Hints

• Which One to Include in Your Kernel .c File?
  • include/asm-i386/xxx.h
  • include/asm/xxx.h
  • include/linux/xxx.h
• During Kernel Configuration
  • include/asm is linked to include/asm-ARCH
• Preferable:
  • #include
  • #include

When Will the Tasklet Run?

• void tasklet_schedule(struct tasklet_struct *t)
  • To schedule a tasklet to run soon
  • Defined In include/linux/interrupt.h:

    static inline void tasklet_schedule(struct tasklet_struct *t)
    {
            if (!test_and_set_bit(TASKLET_STATE_SCHED, &t->state))
                    __tasklet_schedule(t);
    }
    

__tasklet_schedule()

• In kernel/softirq.c

  void __tasklet_schedule(struct tasklet_struct *t)
  {
          int cpu = smp_processor_id();
          unsigned long flags;
  
          local_irq_save(flags);
          t->next = tasklet_vec[cpu].list;
          tasklet_vec[cpu].list = t;
          cpu_raise_softirq(cpu, TASKLET_SOFTIRQ);
          local_irq_restore(flags);
  }
  

Soft IRQ

• In kernel/softirq.c

  void cpu_raise_softirq(unsigned int cpu, unsigned int nr)
  {
          __cpu_raise_softirq(cpu, nr);
          if (!(local_irq_count(cpu) | local_bh_count(cpu)))
                  wakeup_softirqd(cpu);
  }
  

• In include/asm-i386/softirq.h

  #define __cpu_raise_softirq(cpu, nr)                __set_bit(nr, &softirq_pending(cpu))
  

ksoftirqd Thread

• One Per CPU (e.g., ps ax)

      3  ?  SWN    0:03 [ksoftirqd_CPU0]
      4  ?  SWN    0:02 [ksoftirqd_CPU1]
  

• In kernel/softirq.c

  static int ksoftirqd(void * __bind_cpu)
  {
          ...
                  ...
                  while (softirq_pending(cpu)) {
                          do_softirq();
                          ...
                  }
          ...
  }
  

All Connected Together

• do_softirq() calls softirq_vec[h].action()
• softirq_init() calls
  • open_softirq(TASKLET_SOFTIRQ, tasklet_action, NULL);
  • Which equals

    softirq_vec[TASKLET_SOFTIRQ].data = NULL;
    softirq_vec[TASKLET_SOFTIRQ].action = tasklet_action;
    

• tasklet_action():
  • Transverse tasklet_vec[cpu].list
  • And for each struct tasklet_struct *t, do
    • t->func(t->data)

Synchronization Techniques

• Hardware Support
• Atomic Operations
• Disabling Interrupt
• Locking
  • Spin locks
  • Semaphores

Hardware Support

• Fundamentally, Mutual Exclusion Requires Hardware Support
  • Bootstrap from hardware-supported atomic action
• Single Processor
  • cli and sti instruction: disable (and enable) all interrupts
• SMP Architecture
  • The "lock" instruction prefix: lock the memory bus for this instruction (so no other CPU can access the memory until this instruction is done)

Atomic Operations

• Execute a Single Instruction in an "Atomic" Way, Even Under Multiprocessor System
  • Supported by SMP hardware (with the "lock" instruction prefix)
  • Two types: bit ops and atomic integer variable
• Bit Ops: Change a Bit in Any Memory Address
  • In include/asm-i386/bitops.h
    • void set_bit(int nr, volatile void * addr)
    • void clean_bit(int nr, volatile void * addr)
    • int test_and_set_bit(int nr, volatile void * addr)
    • and more

Atomic Integer Variable

• Atomic Operations on Integer Instead of Bit
• Defined in include/asm-i386/atomic.h
• Data Structure: Type atomic_t
• Functions:
  • atomic_read(v)
  • atomic_set(v,i)
  • atomic_add(i,v)
  • atomic_inc(v)
  • ...

Interrupt Disabling

• Disable/Enable All Interrupts in This CPU: Use local_irq_disable() or local_irq_enable()
  • Implemented by cli or sti instruction
  • If SMP system: has no effect on other CPUs
• Global Interrupt Disabling/Enabling: cli() or sti()
  • In uniprocessor system: same as cli and sti instruction
  • SMP: use spin lock to delay interrupt handlers in other CPUs
    • Implemented as __global_cli() and __global_sti()
    • See arch/i386/kernel/irq.c

Saving eflags Register Content

• Must Save Register Content Before Interrupt Disabling and Restore it After Re-enabling
  • Register includes the interrupt flag (IF)
  • See include/asm-i386/system.h
• Local Save/Restore Flags
  • __save_flags(long) and __restore_flags(long)
• Global Save/Restore Flags
  • save_flags(long) and restore_flags(long)
  • SMP: implemented as
    • x = __global_save_flags() and __global_restore_flags(x)
    • See arch/i386/kernel/irq.c

Spin Lock

• A Locking Mechanism for SMP System
  • Through a shared variable
  • Acquire the lock by setting the variable
  • "Spin" in a busy-wait loop until the variable is unset
  • Should be used with care: holding a spin lock too long may cause other CPUs to waste time in busy waiting
• In a Uniprocessor System
  • Implemented as no-op (because it is the only process running)

Spin Lock Implementation

• Defined In include/asm-i386/spinlock.h
• Data type: spinlock_t

  typedef struct {
          volatile unsigned int lock;	// 1: unlocked, <=0: locked
  } spinlock_t;
  

• To lock: spin_lock(lock) macro

  1:	lock; decb (lock->lock)          2: cmpb $0, (lock->lock)
  	js 2f				    rep; nop
  					    jle 2b
  					    jmp 1b
  

• To unlock: spin_unlock(lock) macro

  	movb $1, (lock->lock)
  

Using Spin Lock

• Include <linux/spinlock.h>
• Define spin lock variable:
  • spinlock_t my_lock = SPIN_LOCK_UNLOCKED;
• To lock, call spin_lock(my_lock)
  • Variants:
    • With IRQ disabled: spin_lock_irq(), spin_lock_irqsave()
    • With BH disabled: spin_lock_bh()
• To unlock, call spin_unlock(my_lock)
• To check, spin_is_locked(my_lock) returns 1/0
  • Variants: spin_trylock(), spin_unlock_wait()

Read/Write Spin Lock

• Allow Multiple Readers but Only One Writer
• Data type: rwlock_t
  • In include/asm-i386/spinlock.h
  • Ex: rwlock_t my_lock = RW_LOCK_UNLOCKED;
• Operations:
  • void read_lock(rwlock_t *rw)
  • void read_unlock(rwlock_t *rw)
  • void write_lock(rwlock_t *rw)
  • void write_unlock(rwlock_t *rw)
  • And more (with _irq, _irqsave, _bh, ...)

Kernel Semaphores

• Concept of Semaphore
  • A number of resource available for claimed by task
  • Task put on the wait queue if resource unavailable
  • Task waits up when resource available (released)
• Data Structure
  • In include/asm-i386/semaphore.h

    struct semaphore {
            atomic_t count;	// > 0: available, <=0: busy
            int sleepers;
            wait_queue_head_t wait;
    };
    

More on Kernel Semaphores

• MUTEX: Number of Resource is 1
• To Initialize: sema_init(struct semaphore *, int)
• To Use a Resource: down(struct semaphore * sem)
  • Atomically decrease count
  • Put current on the wait queue if count<0
  • See __down() in arch/i386/kernel/semaphore.c
• To Release: up(struct semaphore * sem)
  • Atomically increment count
  • Wake up one task on wait queue if count<=0
  • See __up() in arch/i386/kernel/semaphore.c

How to Protect Critical Session?

• A Simple Way (Using Interrupt Disabling)

          ...
          unsigned long flags;
          save_flags(flags);
          cli();
          ... critical session ...
          restore_flags(flags);
          ...
  

  • Disadvantage: stops all CPU, really expensive
• The Better Way: Use spin lock!

Next Lecture

• No Class This Friday
• Resume Next Monday
• Device Driver