Pack aware server

News about StepMania Online.
houkouonchi
Site Admin
Posts: 156
Joined: Sun Sep 16, 2012 1:12 pm

Pack aware server

Postby houkouonchi » Tue Apr 02, 2013 1:45 am

Pack scanning support
------UPDATE------
Update: 04-06

I found another bug. The client will use the translated title/artist/subtitle when one is available. I had to re-index the packs/songs and modify the server code to check for that. Once I did that pack detection accuracy went up again and for me I went from 140 packs detected to 155. Please run /scan again if you haven't since this update.

------UPDATE------

------UPDATE------
Update: 04-05

There was a bug in the scanning which didn't handle packs where the unique identifier had special/foreign characters in it (utf8 string). This caused some packs to not be detected. I fixed this problem and now for me it is detecting 140 packs vs 130. I suggest everyone re-run a /scan.

------UPDATE------

------UPDATE------
Pack scanning has been enabled on the regular server.

Run /scan to scan packs. Keep in mind if your far away from the server it can take a minute or two. It must be ran from within a room.

Run /commonpacks to see what packs you have in common with others in the room. Keep in mind as of now only about half of the packs have abbreviations set so you will get comma's with no value as that means the pack has not been assigned a value yet. Also it won't display unless everyone has ran /scan (it will say which user needs to run it when you run /commonpacks).
------UPDATE------

I know this feature has been requested a lot. Originally I didn't think implementing something like this would be possible but this is kind of a hack around not modifying existing game protocol.

So after I spent some time indexing the packs I found that 1150 of the ~1300 packs listed on the site have at least one unique song in them and thus packs can be identified from the song assuming the client has not deleted songs out of their pack folders.

I added a /scan command (currently only on the beta server) that has the server go through the list to figure out what packs you have.

Currently the server just prints what packs it thinks you have.

As soon as I implement it (probably in the next few days or this weekend) the server should be able to store whac packs you have in the db (so just need to /scan when you add packs) and then a /commonpacks command you can run that will be able to check all the users in the room and print out what packs people have in common.

I would appreciate if some people would go on the beta server and test this feature out and see if it looks like the server is getting all (or most) of your packs and report back on how it worked (as well as how slow it was) by commenting on the forum.

It only took around 20 seconds to scan though 1150 packs on my home machine but being <5ms away from the server means it would go much quicker than foreign folks. When the final product is done it likely will only check the top 20-30% of the packs and not all in order to increase scan time.

The beta server as always is port 9999 so connect to:

stepmaniaonline.net:9999

Then make a room and then type /scan

Thanks!
-Houkouonchi

User avatar
blind
Posts: 157
Joined: Mon Oct 01, 2012 3:38 pm
Contact:

Re: Pack aware server

Postby blind » Tue Apr 02, 2013 6:08 am

This is awesome! Will do when I get home.

User avatar
foxfire667
Moderator
Posts: 530
Joined: Sat Sep 29, 2012 5:09 pm
Location: USA
Contact:

Re: Pack aware server

Postby foxfire667 » Tue Apr 02, 2013 7:53 am

I just tried out the scan command, and I'm quite impressed with it's ability to determine what packs I have. I believe throughout the entire scan it made two mistakes, the first one was that it believed I had In the pants (no videos), and the second was it didn't identify one of the SS Styles (I believe it was SS 22nd style). The second mistake is probably due to that particular pack having no unique song or something similar, and I'd imagine the first mistake comes from perhaps the unique song used from that pack also being some sort of single release I might have picked up a few years back. Although considering that I have close to 400 packs installed, it nailed pretty much everything that I had without any serious issues.

According to a quick ping, I'm about 94ms from the server connection wise. For it to scan close to 400 packs, it took about ~3-4 minutes, which I didn't feel was too bad considering the sheer size of my pack set (although compared to you it seems very slow, but as you said, you are so close to the server). I would imagine the average person here on SMO wouldn't have more than 75-100 packs unless they were trying to collect them, so it would probably take less than a minute for most at my speeds.

Putting some suggestions in spoiler tags since they don't directly pertain to the questions you asked.
Spoiler: show
I noticed that at the end of the scan, there are no real details shown by the server in regards to how much was actually recorded. It would be nice to see some statistics show up after scanning (or after successive scans for that matter) for various things. Perhaps something along the lines of:

SMO has found you have X packs installed, and has recorded them.
SMO has found you have installed X more packs since the last scan, and has added them to your list.

Since people might potentially remove packs from their lists over time, I feel that there should be an additional scan command that would force a rescan and remove the current database recordings of a users packs. That way if people have packs they removed, the database doesn't think they still have them, and list them as such to others. It might also be a decent idea for the server to tell users when it has been a while since their last full rescan (like over a month or so) so people remember to do it if they removed packs from their roster (or a quick scan if they just added a few and forgot).
Image

houkouonchi
Site Admin
Posts: 156
Joined: Sun Sep 16, 2012 1:12 pm

Re: Pack aware server

Postby houkouonchi » Tue Apr 02, 2013 8:57 am

Some things to keep in mind. I don]t believe how many packs you actually have should effect scanning time. Even if you have 400 and it shows 400 results its still scanning through the entire 1150 packs it knows about and only prints if it thinks the client has it.

Currently nothing is being stored in the db. The plan was that every time you run /scan it will blow out your current packlist and re-generate so you sould just re-run it when removing or adding packs. I can add final stats at the end as well or possibly do that in a separate command. For now its just a proof of concept where the scan goes and prints results. It won't look/behave like that when its done. Also when it prints packs its likely going to be using abbreviations and not the full pack names. This is just a test/proof of concept in its raw form.

User avatar
foxfire667
Moderator
Posts: 530
Joined: Sat Sep 29, 2012 5:09 pm
Location: USA
Contact:

Re: Pack aware server

Postby foxfire667 » Tue Apr 02, 2013 9:58 am

houkouonchi wrote:Some things to keep in mind. I don]t believe how many packs you actually have should effect scanning time. Even if you have 400 and it shows 400 results its still scanning through the entire 1150 packs it knows about and only prints if it thinks the client has it.

I wasn't aware of this (I really should have been, looking back I have no clue how I could have assumed the server would scan less packs because you have less).

I can see it's just a proof of concept, and that it will be much more refined in the upcoming days. Just made those two suggestions since I didn't see anything regarding them in your main post. Although with the new information, the statistics are pretty much the only valid suggestion (which you say might be added).

But yeah, it looks like it's coming together great, and everything seems to work fine.
Image

User avatar
Izzy
Moderator
Posts: 531
Joined: Thu Sep 27, 2012 3:03 pm
Location: Leawood, Kansas
Contact:

Re: Pack aware server

Postby Izzy » Tue Apr 02, 2013 11:37 am

Going to try this now. Sounds crazy. :shock:

Edit: Added around 100 more pack abbreviations for common packs.

Guilhermeziat
Posts: 116
Joined: Wed Mar 20, 2013 5:16 pm

Re: Pack aware server

Postby Guilhermeziat » Wed Apr 03, 2013 4:14 pm

Hey, as it was scanning my packs, it started choosing random songs idk why
http://prntscr.com/z39t9
Sometimes it changed like 4-5 times in a row.
What's that related to?

Guilhermeziat
Posts: 116
Joined: Wed Mar 20, 2013 5:16 pm

Re: Pack aware server

Postby Guilhermeziat » Wed Apr 03, 2013 4:42 pm

http://prntscr.com/z3dod
I also noticed it stopped scanning after a certain point. When I tried to scan again it said it was already in progress.

houkouonchi
Site Admin
Posts: 156
Joined: Sun Sep 16, 2012 1:12 pm

Re: Pack aware server

Postby houkouonchi » Thu Apr 04, 2013 8:50 pm

Guilhermeziat wrote:http://prntscr.com/z3dod
I also noticed it stopped scanning after a certain point. When I tried to scan again it said it was already in progress.


That is normal. When you scan it figures out what packs by sending the client a selection request like you were going to play it so your client switches to said song. If it is going to songs then that means it is finding packs and is a good thing. If its not switching songs that means that its likely its going through packs where you don't have the song. If you had all the packs it would go through them at a regular interval.

If it says scan is already in progress that usually means what it says (its not done yet) although you should still getting progress updates every 8% (100 packs) until its done. If it completely stopped its likely a bug somewhere. The beta server has a fixed version where /stopscan works properly so it can be stopped and restarted if you run into an issue. You should only have to run /scan when you add or remove packs so it should not be a common thing to do.


Return to “StepMania Online News”

Who is online

Users browsing this forum: No registered users and 1 guest