A network driver ‘framework’

1. A network driver ‘framework’ We construct a ‘skeleton’ module showing just the essential pieces of a Linux network device driver

2. Overview user space kernel space Linux operating system Kernel standard runtime libraries networking subsystem device driver module application program hardware

3. Source-code layout netframe.c #include <linux/module.h> #include <linux/etherdevice.h> … typedef struct { /* driver’s private data */ } MY_DRIVERDATA; char modname[ ] = “netframe”; struct net_device *netdev; my_open() my_stop() The network driver’s “payload” functions my_hard_start_xmit() my_isr() my_get_info() The mandatory module- administration functions my_init my_exit

4. module_init() • This function will execute when the driver is installed in the kernel (‘/sbin/insmod’) • Its role is to allocate and partially initialize a ‘struct net_device’ object for our network interface controller (i.e., hardware device), then “register” that object with the kernel • For ethernet NICs there exists a kernel helper-function that drivers can utilize

5. The ‘key’ statements… typedef struct { /* the driver’s private data */ } MY_DRIVERDATA; struct net_device *netdev; static int __init my_init( void ) { netdev = alloc_etherdev( sizeof( MY_DRIVERDATA ) ); if ( !netdev ) return –ENOMEM; netdev->open = my_open; netdev->stop = my_stop; netdev->hard_start_xmit = my_hard_start_xmit; return register_netdev( netdev ); }

6. module_exit() • This function will execute when the driver is removed from the kernel (‘/sbin/rmmod’) • Its role is to “unregister” the ‘net_device’ structure and free the memory that was allocated during the module’s initialization

7. The ‘key’ statements… struct net_device *netdev; static void __exit my_exit( void ) { unregister_netdev( netdev ); free_netdev( netdev ); }

8. open() • The kernel will call this function when the system administrator “configures” the NIC (e.g., with the ‘/sbin/ifconfig’ command) to assign an IP-address to the interface and and bring it UP • Thus the role of ‘open()’ would be to reset the hardware to a known working state and initiate packet-queueing by the kernel

9. The ‘key’ statements… int my_open( struct net_device *dev ) { /* initialize any remaining ‘private’ data */ /* prepare the hardware for operation */ /* install an Interrupt Service Routine */ /* enable the NIC to generate interrupts */ netif_start_queue( netdev ); return 0; //SUCCESS }

10. stop() • The kernel will call this function when the NIC is brought DOWN (i.e., to turn off its transmission and reception of packets) • This could occur because of a command (such as ‘/sbin/ifconfig’) executed by the System Administrator, or because a user is removing the driver-module from the kernel (with the ‘/sbin/rmmod’ command)

11. The ‘key’ statements… int my_stop( struct net_device *dev ) { netif_stop_queue( netdev ); /* kill any previously scheduled ‘tasklets’ (or other deferred work) */ /* turn off the NIC’s transmit and receive engines */ /* disable the NIC’s ability to generate interrupts */ /* delete the NIC’s Interrupt Service Routine */ return 0; //SUCCESS }

12. hard_start_xmit() • The kernel will call this function whenever it has data that it wants the NIC to transmit • The kernel will supply the address for a socket-buffer (‘struct sk_buff’) that holds the packet-data that is to be transmitted • So this function’s duties are: to initiate transmission, update relevant statistics, and then release that ‘sk_buff’ structure

13. The ‘key’ statements… int my_hard_start_xmit( struct sk_buff *skb, struct net_device *dev ) { /* code goes here to initiate transmission by the hardware */ dev->trans_start = jiffies; dev->stats.tx_packets += 1; dev->stats.tx_bytes += skb->len; dev_kfree_skb( skb ); return 0; //SUCCESS }

14. What about reception? • The NIC hardware receives data-packets asynchronously – not at a time of its own choosing – and we don’t want our system to be ‘stalled’ doing ‘busy-waiting’ • Thus an interrupt handler is normally used to detect and arrange for received packets to be validated and dispatched to upper layers in the kernel’s network subsystem

15. Simulating an interrupt • Our network device-driver ‘framework’ was only designed for demonstration purposes; it does not work with any actual hardware • But we can use a ‘software interrupt’ that will trigger the execution of our ISR • To implement this scheme, we’ll need to employ an otherwise unused IRQ-number, along with its associated ‘Interrupt-ID’

16. Advanced Programmable Interrupt Controller Multi-CORE CPU CPU 0 CPU 1 I/O APIC IRQ0 IRQ1 IRQ2 IRQ3 IRQ23 ● ● ● LOCAL APIC LOCAL APIC The I/O APIC component is programmable – its 24 inputs can be assigned to interrupt ID-numbers in the range 0x20..0xFF (lower numbers are reserved by Intel for the CPU’s exception-vectors) The I/O-APIC’s 24 Redirection Table registers determine these assignments

17. Two-dozen IRQs • The I/O APIC in our classroom machines supports 24 Interrupt-Request input-lines • Its 24 programmable registers determine how interrupt-signals get routed to CPUs Redirection-table

18. Redirection Table Entry 63 56 55 48 32 destination extended destination reserved 31 16 15 14 13 12 11 10 9 8 7 0 reserved M A S K E / L R I R R H / L S T A T U S L / P delivery mode interrupt vector ID 000 = Fixed 001 = Lowest Priority 010 = SMI 011 = (reserved) 100 = NMI 101 = INIT 110 = (reserved) 111 = ExtINT Trigger-Mode (1=Edge-triggered, 0=Level-triggered) Remote IRR (for Level-Triggered only) 0 = Reset when EOI received from Local-APIC 1 = Set when Local-APICs accept Level-Interrupt sent by IO-APIC Interrupt Input-pin Polarity (1=Active-High, 0=Active-Low) Destination-Mode (1=Logical, 0=Physical) Delivery-Status (1=Pending, 0=Idle)

19. Our ‘ioapic.c’ module • Last semester we created a module that will show us which IRQ-numbers are not currently being used by our system, and the Interrupt-IDs those IRQ-signals were assigned to by Linux during ‘startup’ Timeout for an in-class demonstration

20. my_isr() • We created a “dummy” Interrupt Service Routine for our ‘netframe.c’ demo-module #define IRQ 4 // temporarily unused (normally for serial-UART #define intID 0x49 // our I/O-APIC has assigned this ID to to IRQ 4 irqreturn_t my_isr( int irq, void *my_netdev_addr ) { struct net_device *dev = (struct net_device *)my_netdev_addr; MY_DRIVERDATA *priv = dev->priv; // we do processing of the received packet in our “bottom half” tasklet_schedule( &priv->my_rxtasklet ); return IRQ_HANDLED; }

21. Installing and removing an ISR option-flags name for display entry-point for interrupt-handler ISR data-argument IRQ’s signal-number if ( request_irq( IRQ, my_isr, IRQF_SHARED, dev->name, dev ) < 0 ) return –EBUSY; This statement would go in the driver’s ‘open()’ function… …and this statement would go in the driver’s ‘stop()’ function free_irq( IRQ, dev ); Here ‘dev’ is the address of the interface’s ‘struct net_device’ object

22. Processing a received packet • When the NIC notifies our driver that it has received a new ethernet-packet, our driver must allocate a socket-buffer structure for the received data, initialize the ‘sk_buff’ with that data and supporting parameters, then pass that socket-buffer upward to the kernel’s network subsystem for delivery to the appropriate application-program that is listening for it

23. The ‘key’ statements… void my_rxhandler( unsigned long data ) { struct net_device *dev = (struct net_device *)data; struct sk_buff *skb; int rxbytes = 60; // just an artificial value here skb = dev_alloc_skb( rxbytes + 2 ); skb->dev = dev; skb->protocol = eth_type_trans( skb, dev ); skb->ip_summed = CHECKSUM_NONE; dev->stats.rx_packets += 1; dev->stats.rx_bytes += rxbytes; netif_rx( skb ); }

24. Triggering the interrupt… • We allow a user to trigger execution of our interrupt-handler (for testing purposes), by reading from a pseudo-file that our driver creates during module-initialization, whose ‘get_info()’ function includes execution of a software-interrupt instruction: ‘int $0x49’ • This inline assembly language instruction is produced via the GNU ‘asm’ construct

25. Using the ‘asm-construct’ #define intID 0x49 asm(“ int %0 “ : : “i” (intID) ); parameter-value (symbolic) statement keyword parameter type (“i” = immediate data) assembly language opcode parameter indicator This example shows how a symbolic constant’s value, defined in the high-level C programming language using a ‘#define’ preprocessor directive, is able to be referenced by an “inline” assembly language statement within a C code-module

26. Testing our ‘framework’ • You can download, compile, and install our ‘netframe.c’ network driver module • It doesn’t do anything with real hardware, but it does illustrate essential interactions of a network device driver with the Linux operating system’s networking subsystem

27. In-class exercise #1 • Use the ‘/sbin/ifconfig’ command to assign an IP-address to the ‘struct net_device’ object that our framework-module creates • You can discover the interface’s name by using our earlier ‘netdevs.c’ module • You should use a ‘private’ IP-address • EXAMPLE (for station ‘hrn23501’): $ sudo /sbin/ifconfig eth1 192.168.86.1 up

28. In-class exercise #2 • Use ‘ifconfig’ to confirm the IP-address, the IRQ, and the interface’s status: $ /sbin/ifconfig eth1 • Use ‘ifconfig’ to examine the interface’s statistics (packets transmitted/received)

29. In-class exercise #3 • Use the ‘cat’ command to simulate an interrupt from your device’s interface • Verify that your interrupt-handler did get executed, by looking at the statistics, and by displaying the output of a pseudo-file Linux creates (named ‘/proc/interrupts’) $ cat /proc/interrupts

30. In-class exercise #4 • Try removing the ‘netframe.ko’ module (with the ‘/sbin/rmmod’ command), then use the ‘dmesg’ command to see your system’s log-file messages