makeasnek@lemmy.ml to AI@lemmy.mlEnglish · 1 month agoLLM ASICs on USB sticks?lemmy.mlimagemessage-square10fedilinkarrow-up126arrow-down16file-text
arrow-up120arrow-down1imageLLM ASICs on USB sticks?lemmy.mlmakeasnek@lemmy.ml to AI@lemmy.mlEnglish · 1 month agomessage-square10fedilinkfile-text
Source: nostr https://snort.social/nevent1qqsg9c49el0uvn262eq8j3ukqx5jvxzrgcvajcxp23dgru3acfsjqdgzyprqcf0xst760qet2tglytfay2e3wmvh9asdehpjztkceyh0s5r9cqcyqqqqqqgt7uh3n Paper: https://arxiv.org/abs/2406.02528
minus-squarekakes@sh.itjust.workslinkfedilinkarrow-up10·1 month agoNever really occurred to me before how huge a 10x savings would be in terms of parameters on consumer hardware. Like, obviously 10x is a lot, but with the way things are going, it wouldn’t surprise me to see that kind of leap in the next year or two tbh.
Never really occurred to me before how huge a 10x savings would be in terms of parameters on consumer hardware.
Like, obviously 10x is a lot, but with the way things are going, it wouldn’t surprise me to see that kind of leap in the next year or two tbh.