Networking, Lag Compensation, and Hit Detection

[mmacola]

first

Feb 4, 2013 15:59:54 GMT -5 Mousey said:

Now lets say that you have a latency of something nuts like 200 ms.

My regular latency

[markopolo]

I woas going to make some crude Tl;DR joke....

But I did read it....

nice

[APOCALYPSE]

Non-ironic golf clap.

[Slick]

Recommend sticky.

[Zero IX]

Well that sealed the deal

I'm in love with maus

[bucket415]

Feb 4, 2013 17:02:31 GMT -5 Slick said:

Recommend sticky.

[KingVaroon]

What's up with the color scheme? I thought Den wears green and Mousey wears pink. Why is Mousey wearing Den's clothes?

[Usagi]

Cool beans.

[probaddie]

Cheers on the excellent post. Six stars out of six. (Why out of six stars? Because cabbage, that's why.)

[UrbaneVirtuoso]

And I thought those anti-lag compensation videos on CoD4 were enough -- this takes the cake if people hadn't seen it before.

Good job. :3

[mw2baller]

cod4 had the whole pseudo-bullet-travel effect so i'm pretty sure that's what caused a lot of earlier hit detection complaints.

[tiesieman]

you did a pretty damn good job explaining that mus
Hopefully some people will begin realise that there's just no viable workarounds to lag.


Do have a question about gamestates which toys talked about (how it's significantly lower on consoles). If anyone knows, what do these gamestates/rates affect? Buffer times? Delays between the client/server?

[Farslain]

That was an excellent write up, thanks for taking the time to produce it!

Quite a lot of that stuff was new to me, so I thought I'd de-lurk to show my appreciation.

I'm curious what peoples thoughts around here are concerning videos that address lag comp as a 'thing' are, like ovenbakedmuffin's vid I watched a month ago?

[danoski666]

Yargh, this be fine post m8ey.

[Brick2urface]

Aren't you in college? When the fuck did you have time for this?

[bucket415]

Mousey,

Do you stand by your previous statement that the differences we've (COD community) noticed, in regards to the "lag comp" issues from COD4 / W@W / MW2 vs BLOPS / MW3 / BLOPS2 are strictly related to the removal of Stopping Power? Because it looks like you are agreeing that there is a difference in the netcoding.

I can live with a few lag issues. I do understand that they are going to happen in an online game. What I can't really live with is the consistency of the completely bullshit deaths that these lag issues cause. Its like the system changed from the old games, where 90% of the games ran smooth and 10% had lag issues, to now, 90% of the games have lag issues and only 10% run properly.

Has the addition of theater mode, COD casting, the additional tracking of ridiculous stats (distance fallen, average altitude, etc) helped create these new lag problems?

[Lexapro]

Great read, would like some sources and citations though.

You know, before I spread the word.

[Dumien]

Your best post Mouse.

Excellent job. Easy Read.

[timeitself]

Nice post. Thanks for the very through explanation.

I'm guessing the whole blopshot (kill from second chance with primary weapon) was due to the way client side hit detection works. Makes a lot more sense with this info.

But I'm still wondering what the differences are in black ops 2. It certainly feels different. Playing on a three bar feels as bad as a 2 bar from the previous games. I notice many more instances of rewind lag comp. Hearing more than lethal damage fired at an enemy who kills me. Maybe there isn't a difference, maybe there is one but not so much related to lag comp? Any idea why the ammo counter isn't in sync with the firing animation? Can we assign any significance to how long it takes to update?

[Deleted]

Errr I'm not sure if you have this in that huge post, but you could add one thing on that second picture of part 3.0. There was some video floating around this board that showed the broken playercamera. Some guy tested this on split-screen and called it local lag compensation, so you should mention that is not the case. Essentially, the guy who came out of cover from crouch or behind a wall could shoot first without being seen. It could explain some corner bullshit and getting killed by invisible snipers.

[Deleted]

Fucking hell, more technical mumbo-jumbo. Did you learn this on free-time or did you study this? Nicely made post BTW. The pictures really help me to understand some things.

[otisman666]

Wow, even to a common finance guy like myself, this almost makes sense. Really nice write-up. Seriously, A+++

On the down side, I regret to inform you that I have a sneaky suspicions some of the Youtube heroes won't be sending you Christmas cards next year.

[randomguy987]

Great work. The only thing that confused me was where you wrote that hitmarkers are generated server side then sent to clients. I don't know how it works on PC, but I'm 99.9% sure that consoles do hitmarkers client-side. This is easy to see if you use bolt-action or pump-action guns; you get the hitmarker instantly when you shoot, but then the enemy takes another step or two before dropping dead and the +100 coming up on the screen. And it would also explain why you can get (for example) 6+ hitmarkers on an enemy on your screen, yet still not get the kill (I had a final killcam the other night where my FAL got 8 hitmarkers before killing a guy -- in hardcore, no less -- which should be impossible if the server is generating those hitmarkers).

[toysrme]

mousey, what will really blow your mind are the opinions of people that work on it. on their current network model. their opinion is that its great and "can't be improved on due to current hardware constraints". they don't believe that the network hardware, nor CPU has enough overhead to process better. no, not *just* the CPU, but i've even heard a claim that the built in adapters themselves can't handle more. really? that's kinda funny cause 10mb ISA slot ethernet cards seemed to not have trouble running 50/50 default for LAN games...
what? suddenly we move forwards 20 years hardware and 25 years coding wise and no... it's the HARDWARE'S fault...

excuse me? i've previously shown myself running 60/60 snap/tick rates on xbox 360 to no ill effect. That video has been flagged by Activision so many times i had to delete it several times and make it private. and you want to say in public that it's the hardware???

not to mention how many of us were running tick rates of AT LEAST 40's back on ancient PC hardware in the day... sub 1ghz processors that were even more strapped down and like CURRENT console, and like CURRENT console you only ran 30-40fps AVERAGE frame rates. the same realm of frame rates ANY console COD version has ever run. they're simply retarded is the problem.


for examaple. continual claims by programming staff that console COD runs 60fps... excuse me? xbox 360 is faster than ps3, and the highest FPS any COD has ever had in multiplayer is 54fps on average. and that's highrise MW2. about as uncomplicated texture and geometry wise as you can possibly make a map.

Every other online game made doesn't have problems with lag? I could go into great detail as to why this statement is incorrect. However, I'll say that our game runs at 60fps and this compounds the challenge of equalizing the impact of different client to host latency disparity.

Cheers,

G IH A IN ID II

while i was not involved in the conversation, another ximmer probed their responses to the simple issue of dial-up networking rates.
(i washed my hands of developers years ago after butting heads personally with d.v. (he had me XBL profile and console banned by microsoft's infamous thePro. ive had cliffyB look at the look mechanics in unreal games, only for the programmers to dash the fact that they cant program a look mechanic for doo-doo.)
the only way to get a dev to listen to anyone is paying a lot of money in a kickstarter project...




community.callofduty.com/message/205442001#205442001

Ghandi, I know you have a lot people answering my questions, but for the past weeks I have been continually checking if you have responded to my question.

Will you be able to? If it is something you can't answer that's still an answer for me.


Thanks!

community.callofduty.com/message/205442000#205442000

Hey Ghandi!!!!!!

1st off, thanks for answering the communities questions... it really does help alleviate our frustrations with this game! thanks!


However,
I PM'd you this the other day and posted it in this thread, and I was really really hoping you could shine some light on it!


I have wanted to find this out for years now... but COD is based on the old Quake engine, right? obviously updated over the years into what we now have. BUT it's netcoding speed has not been changed for years!... which is why when we look at our network bandwith (via router/tools etc.) being used when playing COD it BARELY uses anything! It is dismal right?

Please please please answer WHY this pipeline has not been doubled, or at least upped?
Does anyone use dial up anymore for gaming....? if so, why should the pipeline be set for the minimum like this?

Why isn't the netcode set higher since XBOX live requires at LEAST DLS.

Wouldn't uppping the bandwidth ammount (even slightly) significantly address many of the problems we are seeing with the network problems in COD?

If you answer this question it will make my day!!!! - because it is extremely frustrating not knowing this answer.

It seems like such an EASY fix for COD, but then there must be a reason why right?

So.

1. Why is the netcode's pipeline set so low?

2. Has auto-aim/aim assist been strengthened since Modern Warfare 2? If so why? It feels like it is way to high - locking onto people behind doors/walls even!

3. Can we PLEASE get an option to completely disable auto-aim aim assist for multiplayer on XBOX 360?

4. Are spawns going to be further fixed soon? Enemies are STILL being spawned right next to me, allowing them to know exactly where I am since I JUST killed them (I am not even tactical loitering), and they get me in the back.


Thanks Ghandi!

community.callofduty.com/message/205442179#205442179

MW3's netcode bears little relation to Quake 3 at this juncture, and hasn't for a long time.

Low payload transfers are good. We constantly strive to minimize packet transfer from both host to client and vice versa. This is a sign of an optimised network engine. Think about what the game is synchronising nowadays compared to games in the past.

Auto aim has not been changed to my knowledge.

There is no plan to disable auto aim.

We are continuing to look at the spawn system.

Cheers,

G IH A IN ID II

community.callofduty.com/message/205442180#205442180

Sorry Zacharybinx34.

Send me it again and I'll answer it if I can.

Edit: I just did. I'm going to sleep now before the kids wake me up in about 3 hours!

Cheers,

G IH A IN ID II

imo,

What they need to do is double the network rate. the hosts & clients need to be sending out 40 updates a second. which would work pretty good considering cod is normally running 40-60fps in MP on PS3/360.

The update rate doesn't really change the network overhead drastically so it could be doubled & be fine for people with 128k connections and up.



P.S.

Want to buy GUN GAME !!! =)

community.callofduty.com/message/205443497#205443497

Hey Ghandi,

Again, thanks for responding to my question! But I have one last follow up question!


Wouldn't doubling the network rate that the hosts & clients need to be sending out up to 40 updates a second fix a lot of the problems? Which would work pretty good considering COD is normally running 40-60fps in MP on PS3/360??? This would only require what...? 128K internet.... who doesn't have that anymore today? 128K is nothing!



Wouldn't this have a very successful impact on many of the problems we are seeing with "HIT DETECTION" and what many are refering to as the "Lag Compensation" problem? (which as you and I know is a broad brush term that is being thrown onto every issue of connectivity!)



I understand the game NEEDS to have some form of lag compensation (the interpolation & extrapolation done to predic what players will do in the future). BUT you guys can tweak things for some improvements, but isn't it true that all the overhead from having to run through XBL and PSN have a dramatic bearing on the networking issues?


Thanks Ghandi! I really appreciate you taking time to talk with the community!

Zach

Ghandi, Thanks for the update, however. Please read/respond to this. I am TRYING to help you guys and I really think what I am suggesting would SUBSTANTIALLY fix much of the problems we are having with LAG.


Wouldn't doubling the network rate that the hosts & clients need to be sending out up to 40 updates a second fix a lot of the problems? Which would work pretty good considering COD is normally running 40-60fps in MP on PS3/360???


Wouldn't this have a very successful impact on many of the problems we are seeing with "HIT DETECTION" and what many are refering to as the "Lag Compensation" problem? (which as you and I know is a broad brush term that is being thrown onto every issue of connectivity!)


I understand the game NEEDS to have some form of lag compensation (the interpolation & extrapolation done to predic what players will do in the future). BUT you guys can tweak things for some improvements, but isn't it true that all the overhead from having to run through XBL and PSN have a dramatic bearing on the networking issues?


Thanks Ghandi! I really appreciate you taking time to talk with the community!

Zach

Ghandi, Thanks for the update, however. Please read/respond to this. I am TRYING to help you guys and I really think what I am suggesting would SUBSTANTIALLY fix much of the problems we are having with LAG.


Wouldn't doubling the network rate that the hosts & clients need to be sending out up to 40 updates a second fix a lot of the problems? Which would work pretty good considering COD is normally running 40-60fps in MP on PS3/360???


Wouldn't this have a very successful impact on many of the problems we are seeing with "HIT DETECTION" and what many are refering to as the "Lag Compensation" problem? (which as you and I know is a broad brush term that is being thrown onto every issue of connectivity!)


I understand the game NEEDS to have some form of lag compensation (the interpolation & extrapolation done to predic what players will do in the future). BUT you guys can tweak things for some improvements, but isn't it true that all the overhead from having to run through XBL and PSN have a dramatic bearing on the networking issues?


Thanks Ghandi! I really appreciate you taking time to talk with the community!

Zach

community.callofduty.com/message/205460574#205460574

Ghandi, please look at my PM's regarding the send/receives between the client/host per second.

I have provided information that what you told me about increasing it equaling drop in FPS on consoles being completely false.


This is a fix that you need to actually look into. Those who mod/hack their consoles have done it for private matches with no FPS drops and the game runs GREAT!

community.callofduty.com/message/205463059#205463059

Ghandi.... brother... please please please fix the lag by doubling the send/receive per second w/ client/host....

As I PM'd you and said numerous times, IT DOESN"T INCREASE FPS ON CONSOLES like you thought it did. Whoever told you that was completely wrong. People with modded 360's (not suggesting ANYONE do this) have done it and have NO PROBLEMS at all with any FPS drops. The boost in netcode performance is spectacular!


Please actually fix the netcode problem.

When can we expect the PS3 netcode changes to come to XBOX 360? The game is terrible right now for multiplayer online on the 360.

community.callofduty.com/message/205463908#205463908

Hello Zach,

In order to put this to bed once and for all.

As it stands, based on the product of message size, update frequency and client count we're butting up against the limits of consumer grade connection capacity - so we can't "just double" the update frequency even if there was negligible encoding cost. There are platform provider requirements that dictate that the game must play on a minimum spec connection.

And encoding isn't free. The server is already taxed in many instances - doubling its encoding workload isn't feasible from a CPU overhead perspective while maintaining 60fps.

Thanks for the suggestion, however, I suggest the posts cease now my friend.

Cheers,

G IH A IN ID II

here's the simple facts you have to know to tear that doo-doo to pieces:
1) console COD has never been a 60fps game. to call it a 60fps game you must load a multiplayer map, go to the edge of a map where there is no geometry/textures to look at & stare at the skybox.
2) console COD normally runs about 52-54fps on x360. mid-high 40's on ps3
3) everyone on the entire planet that has ever used the lagometer in a quake game should recognize that 1-2 frames drawn per update is AWESOME connection. 3 frames per update is "average" while anything over 6 frames per update is your world heavily guessing. in fact, this is known so well that iD put a big fat HEY IDIOT YOU ARE LAGGING color change built into the meter at 6 frames 20 years ago when they built it. aint anybody ever changed that because it's still true... console COD AVERAGES 4-5 frames drawn per update. so you're ALWAYS boarderline as to being heavily lagging by PC standards from *any* era of gaming. assuming you have great internet you're only a sneeze away from being laggy. god forbid you've got average internet... i've got good internet and stay slammed at 4-5 if i'm not hosting even in local play. god help the rest of you that don't...




then you get into frame drops. oh god the network stalls. god forbid anyone calls in a big even in console COD. like a big airstrike. network traffic gets shit on.

[toysrme]

Feb 5, 2013 7:05:28 GMT -5 Farslain said:

That was an excellent write up, thanks for taking the time to produce it!

Quite a lot of that stuff was new to me, so I thought I'd de-lurk to show my appreciation.

I'm curious what peoples thoughts around here are concerning videos that address lag comp as a 'thing' are, like ovenbakedmuffin's vid I watched a month ago?

they're all trashy videos. youd be best served to quit watching them.

[tiesieman]

toys, you the man. Finally understand that rates stuff

[Farslain]

Thanks for the reply and explanation Mousey, much appreciated.

And cheers toys, can't help it mate, always trying to understand more about the game. I didn't agree with a lot of what was said in the video, especially concerning the movement stuff. Always nice to get more opinions.

[bucket415]

If I had a million sub channel and the connections to other million sub channels, I'd make it my mission to get the game changed.

[brutalonslaught]

To be honest, I whinge about lag a lot, but the developers do tweak it and it's never horrendous. I just thank god most of the european community feels the need to put their country of origin on their emblem/clan tag and if not xbox tells me where people are from. When I play the dirty euros I camp, when I play fellow brits I dominate the shit out of them.

I know I'm an extreme because I live next door to the exchange and have a ridiculously good connection. If there was no (or little) lag compensation (MW3 moved to a very liberal interpretation in a patch) then I get host and literally fuck the face off everyone.

[Broadband]

Cool story, bro.