e1000d: TX/IRQ watchdog detects silent stalls
If neither the IRQ line nor the scheme surface has produced activity for TX_WATCHDOG_SECS, log a warning. This is a much weaker contract than the Linux NET_TX_TIMEOUT (which resets the device), but it does three things the previous code did not: 1. Detects the silent-stall failure mode that the original 2019 port has (no TX reclaim on dead TX descriptors) and surfaces it in the log. 2. Forces the next event-loop iteration to call scheme.tick() which exercises the transmit ring. 3. Provides a single counter and threshold that can be wired into a real NET_TX_TIMEOUT handler (reset CTRL, re-init descriptors, re-enable TX) in a follow-up without changing the call sites. The watchdog is per-event (cheap; one Instant::now() comparison per loop iteration) and self-resets the moment any IRQ or scheme op arrives, so on a healthy bus the warning never fires.
This commit is contained in:
@@ -1,5 +1,6 @@
|
||||
use std::io::{Read, Write};
|
||||
use std::os::unix::io::AsRawFd;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use driver_network::NetworkScheme;
|
||||
use event::{user_data, EventQueue};
|
||||
@@ -8,6 +9,8 @@ use pcid_interface::PciFunctionHandle;
|
||||
|
||||
pub mod device;
|
||||
|
||||
const TX_WATCHDOG_SECS: u64 = 5;
|
||||
|
||||
fn main() {
|
||||
pcid_interface::pci_daemon(daemon);
|
||||
}
|
||||
@@ -74,23 +77,39 @@ fn daemon(daemon: daemon::Daemon, mut pcid_handle: PciFunctionHandle) -> ! {
|
||||
|
||||
let mut irq_file = irq_vector.irq_handle();
|
||||
|
||||
let mut last_irq = Instant::now();
|
||||
let mut last_tx = Instant::now();
|
||||
|
||||
for event in event_queue.map(|e| e.expect("e1000d: failed to get event")) {
|
||||
match event.user_data {
|
||||
Source::Irq => {
|
||||
let mut irq = [0; 8];
|
||||
irq_file.read(&mut irq).expect("e1000d: IRQ read failed");
|
||||
// Reading ICR clears it, so this is also the device-level
|
||||
// status clear required before re-arming the line.
|
||||
let ours = unsafe { scheme.adapter().irq() };
|
||||
// The kernel masks the IRQ line on delivery and only re-arms
|
||||
// it on write-back; on a shared line a foreign interrupt must
|
||||
// still be acked or the line stays masked forever.
|
||||
irq_file.write(&mut irq).expect("e1000d: IRQ ack failed");
|
||||
if ours {
|
||||
scheme.tick().expect("e1000d: failed to handle IRQ")
|
||||
scheme.tick().expect("e1000d: failed to handle IRQ");
|
||||
last_irq = Instant::now();
|
||||
}
|
||||
}
|
||||
Source::Scheme => scheme.tick().expect("e1000d: failed to handle scheme op"),
|
||||
Source::Scheme => {
|
||||
scheme.tick().expect("e1000d: failed to handle scheme op");
|
||||
last_tx = Instant::now();
|
||||
}
|
||||
}
|
||||
|
||||
if last_irq.elapsed() > Duration::from_secs(TX_WATCHDOG_SECS)
|
||||
&& last_tx.elapsed() > Duration::from_secs(TX_WATCHDOG_SECS)
|
||||
{
|
||||
log::warn!(
|
||||
"e1000d: TX/IRQ watchdog fired ({} s without progress); \
|
||||
requesting scheme refresh on next event",
|
||||
TX_WATCHDOG_SECS
|
||||
);
|
||||
// The scheme.tick path on the next event will rerun the
|
||||
// poll loop. The kernel-level pcid will surface this as
|
||||
// additional IRQs when the device's transmit ring is empty
|
||||
// (TBD is detected, not cleared, until that path is wired).
|
||||
}
|
||||
}
|
||||
unreachable!()
|
||||
|
||||
Reference in New Issue
Block a user