(This is work in progress)
This are some code snippets from my Operation Black Sun engine. I hope they give some insight into how Newton physics can be synced over Zoidcom. This is not finished yet, as it doesn’t handle joints and it also has some glitches from time to time which I hope to work out. Oh, and it is 2D only, but 3D should be no problem either (I just don’t send the y component of the vectors).
from This forum post
“..., the main limitation [with Newton] is the fact, that you can’t update objects individually. Individual object updates are necessary if you want to keep player and server in sync, while only sending keypresses to the server, instead of absolute position values. This prohibits all sorts of position cheats. If you can live with some lower form of cheat protection (checking incoming values for sanity on the server and then applying them), you can go ahead and just send the client’s current values, no need for the complex movement replicator from example 07.
If you want to do it the right™ way, and only ever let the client send key press events, then read on. First, you should understand what ex07 does and what happens on which system. Then you have to ask yourself, “If I can’t update objects individually, what else can I do to simulate this?”. The server keeps a history of the physical state of all objects of the last x seconds, and also the input which has been applied in each physics update. When I want to update one single object, I restore the state to the time, where the incoming client input should have been applied, apply the input to the client’s object, and then rerun the whole simulation until I reach the time where I was before. Rerunning the simulation includes reapplying the input of other player’s to their respective objects, so that everyone is where he was, except for the influence that the current player’s object took on everyone else. So, if the player is alone, only his object will be moved (this is the individual update that we seek), if he hits someone else in the process, the other player will be out of sync with the server, which is handles by error correction later. It can also happen that client input arrives a bit early, in that case, it is held in a queue until the server simulation reaches the point where the input should be applied.
This was for one single player. Now, you possibly will have more than one player playing, and the system above will kill your perfomance, because for every received player input from any player, the server has to rewind the simulation and recompute the difference, recomputing several time segments multiple times. As this is very inefficient, I created a input queue in the server. All client input is collected there, and once per server tick, the queue is processed. The server time is rewinded to the oldest input timestamp, the input is applied to the correct object, and then the physics are updated again and again, until a time is reached for which another input from another player is waiting. This makes sure that you rewind only once per server tick. Now you also want to limit the amount of time the server is allowed to rewind. When a client lags out for several seconds, errors are expected and so you can drop inputs that are older than what you deem ok. I yet have to finetune this in my code, I think the current maximum rewind is 400 ms, and at the current tickrate this equals to 16 physic updates, which means, per server tick the physic world is updated 16 times max.
This is the main procedure to keep player objects in sync on the player’s machine and the server. The player client also sends his absolute position once in a while so the server can check if they are equal enough. Once an error theshold is reached, the server sends a full state update to the player. The player receives it, rewinds it’s local simulation to the timestamp of the correction, and then simulates back to the current time applying all the input that the player has made, but which has not been acked by the server, yet.
To summarize, instead of updating single objects, I have to rewind and fast forward time on the server in order to apply the player input in the correct moment. If this sounds easy, let me tell you that it’s not. The devil lies very much in the details, and I had to debug this with extensive logs for a long time fixing little bugs and inconstistencies all over the place. It still goes out of sync sometimes and needs to be corrected, and I think there is still a little problem left. Normally corrections should only be necessary when packet loss occurs, but I need them constantly even in local play.”
As stated already, the code is directly from my engine, so you won’t be able to compile it as is. It consists of the following classes:
Times are always in milliseconds since system boot.
void GameBase::processLevel(unsigned int _deltatime ) { m_level->processLevel(_deltatime); m_time += _deltatime; if (m_starttime == 0) m_starttime = ZoidCom::getTime(); }
The relevant parts of the Process() method. Note that the order is crucial and may not be changed:
ZCom_processInput(eZCom_NoBlock); ZCom_processReplicators(getTimeStep()); processClientInput(); processLevel(getTimeStep()); ZCom_processOutput();
When a server gameobject receives input from it’s owner, the GameObject class with give the input to the GameServer with this method:
void GameServer::pushClientInput(int _object_id, LuaObject _input, unsigned int _clienttime, unsigned int _timestamp) { if (_timestamp < m_time - 100) { sys->log("dropping input: too old (%d)", _timestamp - m_time); return; } ClientInput *ci = new ClientInput(); // the client's keyflags stored in the lua table ci->keyflags = _input; ci->timestamp = _timestamp+getTimeStep(); ci->clienttime = _clienttime; // the object this input has to be applied to ci->object_id = _object_id; // insert sorted DATAITEM<ClientInput*> *item = m_clientinput.GetFirst(); while (item) { if (item->GetData()->timestamp>=ci->timestamp) { m_clientinput.InsertBefore(item, ci); return; } item = item->next; } // if not inserted yet if (ci) m_clientinput.Push_Back(ci); #ifdef NETSYNC_DEBUG sys->log("injected input for time: %d", ci->timestamp); #endif }
This is called from the GameServer::Process() method. It traverses the list of collected inputs from clients, rewinds to the time of the oldest input, applies, forward simulates to the next input time, applies the input and so on.
void GameServer::processClientInput() { #ifdef NETSYNC_DEBUG sys->log("---------------------------"); #endif // current time stores the GameServer's time before we start rewinding and forwarding int current_time = getTime(); int tick = -1, lasttick = -1; bool debug = false; #ifdef NETSYNC_DEBUG if (m_clientinput.Size() > 1) { sys->log("xx %d current_time: %d current_state: %d", m_clientinput.Size(), current_time, current_time-getTimeStep()); debug = true; } else sys->log("current_time: %d current_state: %d", current_time, current_time-getTimeStep()); #endif // get oldest input DATAITEM<ClientInput*> *item = m_clientinput.GetFirst(); while (item) { unsigned int timestamp = item->GetData()->timestamp; // input has to be applied in future if (timestamp > current_time) break; // go back to state before timestamp if (tick == -1) { unsigned int newtime = timestamp - (timestamp % getTimeStep()); // m_time is the GameWorld's current time as we are rewinding and forwarding if (newtime < m_time) { if (m_level->getObjectHandler()->activateLastStateBefore(timestamp) == 0) { delete item->GetData(); item = m_clientinput.Delete(item); continue; } m_time = newtime; } } // compute timeslot tick = timestamp / getTimeStep(); // a new tick, advance time if (tick != lasttick && lasttick != -1 && m_time < current_time) { int i = tick - lasttick; while (i--) processLevel(getTimeStep()); lasttick = tick; } else if (lasttick == -1) lasttick = tick; // apply input #ifdef NETSYNC_DEBUG sys->log("apply inputstamp: localtime: %d clienttime: %d", timestamp, item->GetData()->clienttime); #endif GameObject* obj = m_level->getObjectHandler()->getObject(item->GetData()->object_id); if (obj) { obj->applyInput(item->GetData()->keyflags); // check if this is the last input from this player if (obj->getValidationTimestamp() == item->GetData()->clienttime) // and perform the validation and correction sending with the current state obj->validateOwner(); } delete item->GetData(); // delete listitem and get next item = m_clientinput.Delete(item); } // forward until we reach the time which we had before while (m_time < current_time) processLevel(getTimeStep()); assert(m_time == current_time); #ifdef NETSYNC_DEBUG if (debug) sys->log("yyyy"); #endif }
Note that processLevel() will overwrite the stored state of the current time everytime it is called. So when the server rewinds, changes something and then forward simulates, it will store the updated state at the current simulation time. The next rewind will use the already updated data then. (See ObjectHandler::storeState())
Nothing fancy, but order is important again.
void GameClient::Process() { // get updates from network ZCom_processInput(eZCom_NoBlock); ZCom_processReplicators(getTimeStep()); processLevel(getTimeStep()); // give local input to physic/movement replicator playerobject->updateInput(); ZCom_processOutput(); }
This rewinds the client to old times. Requested by GameObject when a correction is received from the server. Corrections like this only appear for GameObjects that are controlled by a player.
unsigned int GameClient::rewindToLastStateBefore(unsigned int _time) { unsigned int time; time = m_level->getObjectHandler()->activateLastStateBefore(_time); if (time != 0) { m_time = time; return time; } return 0; }
Updates Newton world and tells Objecthandler to update.
void Level::processLevel(unsigned int _time) { m_world->update(float(_time)/1000.0f); m_objh->Process(); //if (!m_objh->isServer()) // OgreNewt::Debugger::getSingleton().showLines(m_world); }
Calls Process() on all Gameobjects (which doesn’t do anything relevant here) and stores the complete state (yes, every gamestate is stored)
void ObjectHandler::Process() { GameObjList trashlist; GameObjItem* item = m_objectlist.GetFirst(); while (item) { GameObject *obj = item->GetData(); obj->Process(); item = item->next; } storeState(); }
Stores the complete physical gamestate for the later rewind operations. If there is a state for this timestamp already, only write new phyics data, but don’t alter the stored player inputs (they are needed for the simulation after the rewinds).
bool ObjectHandler::storeState() { GameState *state = NULL; // // first see if we have a state already // unsigned long servertime = m_game->getTime(); DATAITEM<GameState*> *item = m_statehistory.GetLast(); if (item && servertime < item->GetData()->getTime()) { // find the state then item = m_statehistory.GetFirst(); while (item && item->GetData()->getTime() != servertime) item = item->next; assert(item && "No state for storing"); state = item->GetData(); // store without overwriting player input if (!state->fromObjectHandler(this, servertime, true)) return false; } else { state = new GameState(); // store including player input if (!state->fromObjectHandler(this, servertime, false)) return false; m_statehistory.Push_Back(state); // delete old entries DATAITEM<GameState*> *item = m_statehistory.GetFirst(); while (item) { state = item->GetData(); if (m_game->getTime() - state->getTime() > m_historytime) delete state; else break; m_statehistory.Delete(item); item = m_statehistory.GetFirst(); } } #ifdef NETSYNC_DEBUG sys->log("store: %d", servertime); #endif return true; }
Activate the last state that is before a certain timestamp. That is, search state history and replace current state with the state from the history.
unsigned int ObjectHandler::activateLastStateBefore(unsigned int _timestamp) { DATAITEM<GameState*> *item = m_statehistory.GetLast(); while (item && item->GetData()->getTime() >= _timestamp) item = item->prev; assert(item && "No state to activate"); if (!item) return 0; #ifdef NETSYNC_DEBUG sys->log("activating: %d", item->GetData()->getTime()); #endif item->GetData()->toObjectHandler(this); return item->GetData()->getTime(); }
Setups network stuff.
bool GameObject::Setup() { // make lua instance m_luaobj = m_class->makeInstance(this); m_luadata = tolua_getinstancetable(m_luaobj.GetState(), this, "GameObject"); // // set up networking // m_netnode = new ZCom_Node(); m_netnode->setEventNotification(true, false); m_netnode->beginReplicationSetup(m_class->getReplicationItemsCount()); // movement replicator setup m_moverep = new ZCom_Replicate_Physics(13, ZCOM_REPFLAG_MOSTRECENT, ZCOM_REPRULE_OWNER_2_AUTH|ZCOM_REPRULE_AUTH_2_ALL); m_moverep->setUpdateListener(this); m_netnode->addReplicator(m_moverep, true); // other stuff omitted m_netnode->endReplicationSetup(); // back link from node to object m_netnode->setUserData(this); m_netnode->registerNodeDynamic(m_class->getZComID(), m_handler->getGame()); return true; }
Called a bit after Setup() I think, just basic Newton object setup wrapped through OgreNewt.
bool GameObject::setupPhysics() { // // setup physics // OgreNewt::World* world = m_handler->getGame()->getLevel()->getWorld(); Vector3 collsize(m_class->getMeshRadius()); Vector3 minsize(10.0f); collsize.makeCeil(minsize); // TODO: don't create collision until we have position OgreNewt::Collision *col = NULL; if (m_class->getMeshRadius() == 0.0f) col = new OgreNewt::CollisionPrimitives::Null(world); else col = new OgreNewt::CollisionPrimitives::Ellipsoid(world, collsize); m_body = new OgreNewt::Body(world, col); delete col; new OgreNewt::PrebuiltCustomJoints::Custom2DJoint(m_body, Vector3::UNIT_Y); m_body->setCustomForceAndTorqueCallback<GameObject>(&GameObject::physicForceCallback, this); m_body->setAutoactiveCallback<GameObject>(&GameObject::autoActivateCallback, this); m_body->setPositionOrientation( Ogre::Vector3(-100,0,-100), Quaternion::IDENTITY ); m_body->setAutoFreeze(0); m_body->setAngularDamping(Vector3(0.5)); m_body->setUserData(this); m_body->setMaterialGroupID(m_class->getMaterial()); return true; }
unsigned int GameObject::getValidationTimestamp() { return m_validationtime; }
When ownerclient sends his input, he always sends a validation along, in the form of a position vector. This validation is meant to be checked at a specific time and this function performs the check. The validation info is stored in the time between receival and validateOwner() call.
void GameObject::validateOwner() { if (m_validationtime == 0) return; Vector3 pos = getPosition(); Ogre::Vector3 clipos(m_validationpos.x, 0, m_validationpos.z); Ogre::Vector3 srvpos(pos.x, 0, pos.z); Ogre::Vector3 diff = srvpos - clipos; #ifdef NETSYNC_DEBUG sys->log("%d processing validation: srv (%0.2f,%0.2f) cli(%0.2f,%0.2f) diff(%0.2f) from clienttime: %d", this->m_player->c_id, srvpos.x, srvpos.z, clipos.x, clipos.z, (diff).length(), m_validationtime); #endif if ((diff).length() > 16) { sys->log("validation exceeds treshold, correcting: srv (%0.2f,%0.2f) cli(%0.2f,%0.2f) diff(%0.2f)", srvpos.x, srvpos.z, clipos.x, clipos.z, (diff).length()); sendCorrection(m_validationconn, m_validationtime); } m_validationtime = 0; }
This is called when the validation check fails (i.e. error is too large). It generates a correction and sends it to the player.
void GameObject::sendCorrection(ZCom_ConnID _connid, zU32 _timestamp) { // lock correction sending for ping time if (ZoidCom::getTime() - m_correction_time < (200)) return; Vector3 pos = getPosition(); Vector3 vel = getVelocity(); Vector3 omega = getOmega(); float rot = getRotation(); ZCom_BitStream *state = new ZCom_BitStream(); state->addFloat(pos.x, 23); state->addFloat(pos.z, 23); state->addFloat(vel.x, 23); state->addFloat(vel.z, 23); state->addFloat(omega.y, 23); state->addFloat(rot, 23); m_moverep->sendCorrection(_connid, _timestamp, state); m_correction_time = ZoidCom::getTime(); #ifdef NETSYNC_DEBUG sys->log("sending correction to %d for time %d", _connid, _timestamp); #endif }
This is called on the client by the input system (somehow). The player class (which is not shown) gets the inputflags from the input system and encodes it into a bitstream, ready for transmission. This bitstream is given to updateInput(), which will give it to the custom physics replicator. It also generates a 2nd bitstream containing the validation data (object’s position, so server can check for errors)
void GameObject::updateInput(ZCom_BitStream* _bs) { if (m_netnode->getRole() == eZCom_RoleOwner) { Ogre::Vector3 pos; Ogre::Quaternion ori; m_body->getPositionOrientation(pos, ori); ZCom_BitStream *validation = new ZCom_BitStream; validation->addFloat(pos.x, 23); validation->addFloat(pos.z, 23); #ifdef NETSYNC_DEBUG sys->log("issuing inputupdate with validation cli(%0.2f,%0.2f) omega(%0.2f) rot (%0.2f)", pos.x, pos.z, omega.y, rot); #endif m_moverep->updateInput(validation, _bs); } }
Callback called by Newton to update the object with forces. This applies the forces that come from the player input and are stored in m_force. That is, for each tick, the Newton world is updated and everytime this function here is called for all objects by Newton. The forces stored in m_force are generated by the applyInput() method.
void GameObject::physicForceCallback(OgreNewt::Body* _body) { Ogre::Vector3 pos; Ogre::Vector3 inertia; Ogre::Quaternion ori; Ogre::Real mass; float scalefactor = 30.0f / float(m_handler->getGame()->getTickRate()); #ifdef NETSYNC_DEBUG if (m_netnode->getRole() == eZCom_RoleAuthority) sys->log("-"); else sys->log("- cli %0.2f", m_force.z); #endif // rotation dampening if (m_rotationdamp != 1.0f) { Ogre::Vector3 omega = _body->getOmega(); omega /= ((m_rotationdamp-1.0f)*scalefactor)+1.0f; _body->setOmega(omega); } _body->getMassMatrix(mass, inertia); _body->addForce(Vector3(m_force.x*mass*scalefactor, 0, m_force.z*mass*scalefactor)); _body->addTorque(Vector3(0, m_torque*mass*scalefactor, 0)); }
Called by GameServer::processClientInput(). This gives the input parameters to a Lua function, which will then act on it.
void GameObject::applyInput(LuaObject& _inputtable) { try { // pass input controls to lua LuaObject lfun_applyinput = m_luadata.GetByName("applyInput"); if (!lfun_applyinput.IsFunction()) return; LuaCall call = lfun_applyinput; call << LuaArgToluaType(this, "GameObject") << _inputtable << LuaRun(0); } catch (LuaException &e) { sys->log(LOG_ERROR, "Lua: %s", e.GetErrorMessage()); } }
The Luacode being called is currently this:
-- called by engine when player or AI input needs to be processed function ShipBase:applyInput(input) local angle = self:getDirection() local torque = 0 if input.Turnleft then torque = self.objProps.torque_power end if input.Turnright then torque = -self.objProps.torque_power end self:setTorque(torque) local thrust = 0 if input.Accel then thrust = self.objProps.thrust_power end if input.Backthrust then thrust = -self.objProps.backthrust_power end self:setDirectionalForce(angle, thrust) end
The self:setDirectionalForce() call will set m_force in the GameObject, which in turn is used in the physicForceCallback().