wafer.space Community - fpgas-online

5:46 a.m.

3Bv1.2 ?? has no PoE header

5:50 a.m.

PoE hat powreed v4 setup screen

something is going on. net/nfs boot, pi4 sends the bitstream to the arty then reboots a minute later

9:33 a.m.

same pi with an sd card, brings up a gui desktop, no reboot

9:34 a.m.

I have trouble believing te arty is drawing so much power the pi reboots

@carlfk - Could you check on the Acorns in the Compute Blade?

Tim 'mithro' Ansell

@carlfk - Could you check on the Acorns in the Compute Blade?

They are still there

11:31 p.m.

they are wired to the switch, I could probably figure out which ports by turning them all on

11:32 p.m.

I'll be at ps1 in.. 18 hours ish.

11:32 p.m.

what sort of checking were you looking for?

  From outside, at 10:01 today
  - The index lists nine boards: pi2, pi3, pi5, pi7, pi9, pi11, pi13, pi21 and pi23. None is an Acorn host.
  - The Compute Blade hosts have no page: pi14.html, pi16.html, pi18.html and pi20.html all answer 404.
  - ps1 has no fleet registry page (/fleet/ answers 404), so no boot-check result is visible from outside.

  Last direct look, 31 August (from my notes, not re-checked)
  ┌──────┬──────────┬────────────────────────────────────────────────┐
  │ Host │  Module  │                  On PCIe then                  │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi14 │ CM4      │ Acorn CLE-101, PCI ID 1e24:0101                │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi16 │ CM5 Lite │ Acorn CLE-101, PCI ID 1e24:0101                │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi18 │ CM4      │ nothing, M.2 slot empty                        │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi20 │ CM5 Lite │ a Xilinx card showing the generic ID 10ee:7011 │
  └──────┴──────────┴────────────────────────────────────────────────┘

  All four blades were up that day.

  Not known
  - Whether those blades are powered today, what design is on the cards, or whether our boot check has ever run there.

nice - I can power on those 4 ports

11:38 p.m.

@Tim 'mithro' Ansell all 4 were off. I just powered them all on.

@carlfk - I'll have some follow up instructions on how to "verify" they are wired up correctly.

@Tim 'mithro' Ansell well.. I see 2 of them booting linux:

11:43 p.m.

('10.21.0.116', 6665) [ 9.991035] audit: type=1130 audit(1791157300.039:11): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=modprobe@fuse comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

11:44 p.m.

that same IP seems to have just rebooted

11:44 p.m.

i am wondering if some startup script is causing the reboot

11:45 p.m.

for sure I had a hack because something (camera process I think) didn't have a hostname (needed to tell nginx where video data was coming from )

11:46 p.m.

managing hostname has been ... tricky.

sigh. somehow 16 has become stable

12:00 a.m.

pi@pi16:~ $ uptime 00:59:28 up 6 min, 1 user, load average: 1.42, 1.10, 0.53

12:00 a.m.

120 is still rebooting

12:01 a.m.

120 = 10.21.0.120 = port 20

@carlfk - https://docs.fpgas.online/en/latest/verify/fpgas-verify.html#installing

Tim 'mithro' Ansell

@carlfk - https://docs.fpgas.online/en/latest/verify/fpgas-verify.html#installing

systemctl cat fpgas-verify

12:48 a.m.

EnvironmentFile=-/etc/default/fpgas-verify

12:48 a.m.

cat: /etc/default/fpgas-verify: No such file or directory

12:48 a.m.

that seems wrong

12:49 a.m.

but it might not matter

12:49 a.m.

ExecStart=/usr/bin/fpgas-verify $FPGAS_VERIFY_ARGS

12:54 a.m.

@Tim 'mithro' Ansell other than 'fail' I'm not sure how to read this

carlfk

@Tim 'mithro' Ansell other than 'fail' I'm not sure how to read this

ssh -p 11622 pi@ps1.fpgas.online

welp, I took the reboot out whenever this happened https://github.com/CarlFK/pici/commit/383b11a43ad8f6ff5c3dde772c69eacdec9a486c

re-run dhclient instead of rebooting. might be better? · CarlFK/p...

gah. 20 finely deiced not to reboot

2:56 a.m.

pi@pi20:~ $ uptime 03:55:35 up 2:21, 1 user, load average: 1.00, 1.01, 1.07

Tim 'mithro' Ansell

  From outside, at 10:01 today
  - The index lists nine boards: pi2, pi3, pi5, pi7, pi9, pi11, pi13, pi21 and pi23. None is an Acorn host.
  - The Compute Blade hosts have no page: pi14.html, pi16.html, pi18.html and pi20.html all answer 404.
  - ps1 has no fleet registry page (/fleet/ answers 404), so no boot-check result is visible from outside.

  Last direct look, 31 August (from my notes, not re-checked)
  ┌──────┬──────────┬────────────────────────────────────────────────┐
  │ Host │  Module  │                  On PCIe then                  │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi14 │ CM4      │ Acorn CLE-101, PCI ID 1e24:0101                │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi16 │ CM5 Lite │ Acorn CLE-101, PCI ID 1e24:0101                │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi18 │ CM4      │ nothing, M.2 slot empty                        │
  ├──────┼──────────┼────────────────────────────────────────────────┤
  │ pi20 │ CM5 Lite │ a Xilinx card showing the generic ID 10ee:7011 │
  └──────┴──────────┴────────────────────────────────────────────────┘

  All four blades were up that day.

  Not known
  - Whether those blades are powered today, what design is on the cards, or whether our boot check has ever run there.

CM4 the 4 is for Pi v4 , right?

@carlfk - I'm getting a bunch of stuff for you for these boards.

Oct 04 22:46:59 kas1 dnsmasq-tftp[1228]: sent /srv/tftp/4e45f174/kernel8.img to 10.21.0.118

4:14 a.m.

tftp sends to the 2 CM4x and they don't seem to do anything after that

4:14 a.m.

the 2 CM5s start sending kernel boot messages

omg - 14 and 18 just booted all the way!

7:29 p.m.

and 16 went to sleep.

7:30 p.m.

@Tim 'mithro' Ansell these boards (or maybe just CM?) are super weird

14 18 20 up. 16 boot looping

16 woke uj. all 4 are up. I don't understand.

@carlfk - I think Claude might have fixed it.....

Claude fixed my server?

@carlfk - I'm not 100% happy with the output but this is what it wrote for you at the moment - https://ten64.welland.mithis.com/~tim/ps1-compute-blades-draft9-Letter.pdf

Tim 'mithro' Ansell

@carlfk - I think Claude might have fixed it.....

what did claud fix?

arg. none of the pi are staying booted.

7:21 p.m.

nmap shows 22 open for a second, then gone and the server starts sending boot files again.

amazing - the cm blade at 20 is staying up

and now 16.

8:32 p.m.

none of the Pi v3 or v4 Arty kits. just the 2 blades

03 booted from an SD card but PoE power and is solid, so hard to say it is a power problem

@carlfk - What is the state of the arty boards at ps1?

10:26 p.m.

@carlfk - Are you still at PS1?

I am not there yet, but just finished liuch so now I am getting ready to go

10:27 p.m.

state: nmap shows 22 open on 7 9 16 20. 3 too, but that doesn't count as it is booted from a stock SD RaspiOS

10:28 p.m.

If I power cycle 7 9 16 20 I expectr them to reboot a few 100 times and the maybe stay booted

10:28 p.m.

I have no idea why that happens

10:29 p.m.

oh hey, 7 and 9 just went away. just 16 and 20 now.

10:29 p.m.

summery: big mess

on my way - eta 15m

I have arrived.

Make it stop rebooting!!!

1:50 a.m.

SD card fine. pull SD and let ti netboot, loop forever

@Tim 'mithro' Ansell I am at ps1. about done here. did you want me to check on anything

carlfk

@Tim 'mithro' Ansell I am at ps1. about done here. did you want me to check on anything

leaving ps1 heading home. probably back in 18 hours.

the 4 blade pis are up, and one pi: Raspberry Pi 3 Model B Plus Rev 1.3pi@pi7

carlfk

the 4 blade pis are up, and one pi: Raspberry Pi 3 Model B Plus Rev 1.3pi@pi7

What does the RPi 3b+ have?

Tim 'mithro' Ansell

What does the RPi 3b+ have?

hjave? like it hse the cheep POE dongle?

@carlfk - No FPGA wise - an arty?

yes arty

9:58 p.m.

I swapped x86 server, rebuild the os/stuff. didn't really touch the switch/cables/pi setup -

@carlfk - I'm still not happy with the instructions the AI has written for you - https://ten64.welland.mithis.com/~tim/ps1-compute-blades-copy24-Letter.pdf

and regardless I don't understand how what is happening is possible

Tim 'mithro' Ansell

@carlfk - I'm still not happy with the instructions the AI has written for you - https://ten64.welland.mithis.com/~tim/ps1-compute-blades-copy24-Letter.pdf

I skimmed that - I dont think that will help wiht the current reboot almost forever problem

@carlfk - I have not been looking at the reboot almost forever problem. Did you want me to look into that?

yes

10:05 p.m.

I am about to faf around with https://github.com/systemd/python-systemd#usage to try and get a clue from journalctl --file=/var/log/journal/remote/remote-10.21.0.103.journal

GitHub - systemd/python-systemd: Python wrappers for systemd functi...

Python wrappers for systemd functionality. Contribute to systemd/python-systemd development by creating an account on GitHub.

10:06 p.m.

which is the system logs from 10.21.0.103

10:07 p.m.

all the pi have power. I have no idea why onlye some of them dont log

j = journal.Reader(files=["/var/log/journal/remote/remote-10.21.0.103.journal"])

10:43 p.m.

print(entry['MESSAGE'])

10:43 p.m.

unit_file_build_name_map: normal unit file: /usr/lib/systemd/system/poweroff.target

10:44 p.m.

poweroff is specious :p

.get_previous() looks useful

@Tim 'mithro' Ansell are you / claud logged into ps1 box? someone is. I'm trying to test _netdev things

carlfk

@Tim 'mithro' Ansell are you / claud logged into ps1 box? someone is. I'm trying to test _netdev things

I don't think so?

Tim 'mithro' Ansell

I don't think so?

huh. something was doing stuff. didn't look human. shrug.

10:57 p.m.

@Tim 'mithro' Ansell do you have code to detect if the pi has a camera? like the blades don't so the systemd cam.service thrashes and fills up the logs

Tim 'mithro' Ansell

I don't think so?

ssh -p 10322 pi@ps1.fpgas.online it happened again. pop in and see the bash history

3:00 p.m.

I'm wondering if someone elses AI is .. AIing around

https://paste.debian.net/hidden/73216aa6 bash_history