a thoughtful web.
Good ideas and conversation. No ads, no tracking.   Login or Take a Tour!
comment by mk
mk  ·  4246 days ago  ·  link  ·    ·  parent  ·  post: "unknown or expired link"

Yes, the fnids are the culprit, and they are user-specific continuation functions. There's a rich history of discussion about them on HN. Aside from engineering our way out of them, we can extend the number that can be cached before being culled, or we can decrease the instances of them. We can do both. They are over utilized on Hubski. In my early days, I would generate them willy-nilly, now it's my goal to wind them down to a minimum.

I'll bump up the number, and start picking some low hanging fruit.





thundara  ·  4245 days ago  ·  link  ·  

Hoping that's not a security hole if I spoof a random fnid that belongs to someone else...

Wouldn't it make more sense to couple the storage time of the functions to a specified timeout? Or alternatively, couple one ID to a page, so you don't generate one for each comment for each view? As is, it seems like a 10x increase in traffic would mean a 10x decrease in the time before the fnids expire.

It might cause memory issues, but that could be easily fixed by firing up more servers (Not sure what your opinion is on The Cloud)

mk  ·  4245 days ago  ·  link  ·  

    Hoping that's not a security hole if I spoof a random fnid that belongs to someone else...

https://news.ycombinator.com/item?id=639976

    Wouldn't it make more sense to couple the storage time of the functions to a specified timeout?

Some of the fnids do have timeouts, and that's one of a number of ways to alleviate this. We are currently "in The Cloud", on a Ubuntu server at Digital Ocean. We used to be on Linode, which has great service, but DO has SSD, and that has huge advantages with Hubski's unusual architecture.

thundara  ·  4245 days ago  ·  link  ·  

Both awesome and worrisome. No post from pg on there, did they patch HN (Slash how did they) to fix the security hole? Does hubski have the same fix applied? Dunno if increasing the length of the fnid / cookie would help if your PRNG has been compromised...

mk  ·  4245 days ago  ·  link  ·  

Yeah, really impressive. Not sure about HN, but we use the Arc3 rand-string.

pg did respond.

I'm still looking into what else might have been a part of it.

kleinbl00  ·  4246 days ago  ·  link  ·  

What is the number? That would be a place to start...

mk  ·  4245 days ago  ·  link  ·  

20k atm. I ramped it up in the past, but we brought it down back when we were on the old server and having memory issues. I can increase it, but reducing the load needed scales better.

kleinbl00  ·  4245 days ago  ·  link  ·  

What is that in minutes?

mk  ·  4245 days ago  ·  link  ·  

The culling depends on the rate at which they are generated. I killed some off this morning with a new update, and changed a couple of parameters to reduce the filling rate. I'm going to get to work on some replacement options for some other places.

kleinbl00  ·  4245 days ago  ·  link  ·  

K great. I think it's that "you have no idea how long you have" part that's particularly annoying.

mk  ·  4245 days ago  ·  link  ·  

Perhaps a "This fnid will destruct in X seconds." message.

kleinbl00  ·  4245 days ago  ·  link  ·  

Fire it up when I start typing into the reply box. If I start typing and the fnid has already expired, it gives me a "link has expired, click to refresh" button instead of a "reply" button.