1, 'A' => 2, 'C' => 3, 'B' => 4, 'M' => 5, 'H' => 6, 'Z' => 7 ); /** * @var array */ protected $typeFieldsToMatch = array( 'cylinders', 'cubicCapacity', 'kilowatts', 'bodyType', 'doors' ); /** * @var array */ protected $numericFields = array( 'cylinders', 'cubicCapacity', 'kilowatts', 'doors' ); /** * @var array */ protected $makes = array(); /** * @var array */ protected $models = array(); public function __construct(EntityManager $em, DataStore &$store) { $this->em = $em; $this->util = new ImportUtility($this->em); $this->store = $store; } #-------------------------------------------------------------------------- SECTION /** * @param $fileName */ public function executeOfflineTransunionDataImport($fileName) { if ($this->simulateExecution) { error_log('php /var/www/B4C2/public/index.php vehicle import ' . $fileName . ' >>/log/php.log &'); } else { exec( 'php /var/www/B4C2/public/index.php vehicle import ' . $fileName . ' >>/log/php.log &' ); } } #-------------------------------------------------------------------------- SECTION /** * Ensure that synchronization versions have a full sequence. * return null */ public function ensureSynchronizationVersionSequence() { #-> Get current latest versions. $makeVersion = $this->util->getLatestSynchVersion('Stock\\Entity\\Make'); $modelVersion = $this->util->getLatestSynchVersion('Stock\\Entity\\Model'); $typeVersion = $this->util->getLatestSynchVersion('Stock\\Entity\\Type'); #-> Establish highest current version. $maxVersion = $this->getMaxVersion($makeVersion, $modelVersion, $typeVersion); #-> Where do we need updates? if ($maxVersion > $makeVersion) { $this->updateSynchVersionSequence('Stock\\Entity\\Make', $makeVersion + 1, $maxVersion); } if ($maxVersion > $modelVersion) { $this->updateSynchVersionSequence('Stock\\Entity\\Model', $modelVersion + 1, $maxVersion); } if ($maxVersion > $typeVersion) { $this->updateSynchVersionSequence('Stock\\Entity\\Type', $typeVersion + 1, $maxVersion); } } /** * @param $makeVersion * @param $modelVersion * @param $typeVersion * @return mixed */ public function getMaxVersion($makeVersion, $modelVersion, $typeVersion) { $maxVersion = $makeVersion > $modelVersion ? $makeVersion : $modelVersion; $maxVersion = $maxVersion > $typeVersion ? $maxVersion : $typeVersion; return $maxVersion; } /** * @param string $entityName * @param integer $startVersion * @param integer $endVersion * @throws \Exception */ public function updateSynchVersionSequence($entityName, $startVersion, $endVersion) { #-> Get entries we can update. $numEntriesNeeded = ((int)$endVersion) - ((int)$startVersion) + 1; $entries = $this->em->getRepository($entityName) ->findBy( array('updateVersion' => null), array('id' => 'ASC'), $numEntriesNeeded, 0 ); $this->store->version = ((int)$startVersion) + 1; if ($numEntriesNeeded != count($entries)) { throw new \Exception( 'Cannot synchronize data versions on ' . $entityName . '. Not enough entries available.' ); } foreach ($entries as $entry) { $entry->updateVersion = $this->store->version; $this->store->version++; } $this->em->flush(); } #-------------------------------------------------------------------------- SECTION /** * MAKE THIS PUBLIC!!!!!! * @param array $packet * @return array */ public function cleanupNumericInputFields(array $packet) { foreach ($this->numericFields as $field) { if (array_key_exists($field, $packet)) { $packet[$field] = ('' == $packet[$field]) ? null : $packet[$field]; } } return $packet; } #-------------------------------------------------------------------------- SECTION /** * MAKE THIS PUBLIC!!!!!! * @param array $packet * @return mixed */ public function createOrUpdateVehicleMakeEntry(array $packet) { if (!isset($this->makes[$packet['Make']])) { $makeId = $this->getOrCreateMakeEntry($packet); $this->makes[$packet['Make']] = $makeId; } return $this->makes[$packet['Make']]; } /** * MAKE THIS PUBLIC!!!!!! * @param array $packet * @return mixed */ public function getOrCreateMakeEntry($packet) { $makeId = $this->util->getId( 'Stock\\Entity\\Make', array( 'name' => $packet['Make'] ) ); if (false == $makeId) { $makeId = $this->util->createEntry( 'Stock\\Entity\\Make', array( 'name' => $packet['Make'], 'createVersion' => $this->store->version, 'updateVersion' => $this->store->version ) ); $this->store->numItems++; return $makeId; } return $makeId; } #-------------------------------------------------------------------------- SECTION /** * MAKE THIS PUBLIC!!!!!! * @param array $packet * @param $makeId * @return mixed * @throws \Doctrine\ORM\ORMException */ public function createOrUpdateVehicleModel(array $packet, $makeId) { if (!isset($this->models[$makeId]) || !isset($this->models[$makeId][$packet['Model']])) { $modelId = $this->getOrCreateModelEntry( $packet, $makeId ); $this->models[$makeId][$packet['Model']] = $modelId; } return $this->models[$makeId][$packet['Model']]; } /** * @param array $packet * @param $makeId * @return mixed */ public function getOrCreateModelEntry(array $packet, $makeId) { if (!isset($this->models[$makeId])) { $this->models[$makeId] = array(); } $modelId = $this->util->getId( 'Stock\\Entity\\Model', array( 'make' => $makeId, 'name' => $packet['Model'] ) ); if (false == $modelId) { $modelId = $this->util->createEntry( 'Stock\Entity\Model', array( 'make' => $this->em->getReference('Stock\\Entity\\Make', $makeId), 'name' => $packet['Model'], 'createVersion' => $this->store->version, 'updateVersion' => $this->store->version ) ); $this->store->numItems++; return $modelId; } return $modelId; } #-------------------------------------------------------------------------- SECTION /** * @param array $packet * @param $modelId * @return mixed * @throws \Doctrine\ORM\ORMException */ public function createOrUpdateVehicleType(array $packet, $modelId) { #-> Do data preparations. $packet['name'] = $packet['Variant']; $packet = $this->prepareAdditionalVehicleTypeData($packet); $years = $this->buildYearMapArray(); $packet = $this->prepareDateVehicleTypeData($packet, $years); /** * Here we ensure that we remove spaces and retain leading zeroes to prevent data duplication. * Previously, this field was cast to int which lead to duplications. - FP */ $packet['mmCode'] = (string)$packet['MMCode']; $packet['mmCode'] = preg_replace('/\s+/', '', $packet['mmCode']); #-> Find entry. $type = $this->util->getEntry('Stock\\Entity\\Type', array( 'model' => $modelId, 'mmCode' => $packet['mmCode'] )); if (false == $type) { $packet = $this->prepareVehicleTypeDataForNewEntry($packet, $modelId); $this->util->createEntry('Stock\\Entity\\Type', $packet); $this->store->numItems++; } else if ($this->vehicleTypeUpdateIsNeeded($packet, $type)) { $type->updateVersion = $this->store->version; $this->em->flush($type); $this->store->numItems++; } return $packet; } /** * @param array $packet * @return array */ public function prepareAdditionalVehicleTypeData(array $packet) { $packet['cylinders'] = $packet['NoCylinders']; $packet['cubicCapacity'] = $packet['CubicCapacity']; $packet['kilowatts'] = $packet['Kilowatts']; $packet['bodyType'] = $packet['BodyType']; $packet['doors'] = $packet['NoOfDoors']; return $packet; } /** * @return array */ public function buildYearMapArray() { $nextYear = ((int)date('Y')) + 1; $years = array(); $i = 1; for ($y = 1970; $y < $nextYear; $y++) { $years[$y] = $i; $i++; } return $years; } /** * @param array $packet * @param array $years * @return array * @throws \Doctrine\ORM\ORMException */ public function prepareDateVehicleTypeData(array $packet, $years) { list($introMonth, $introYear) = explode('/', $packet['IntroDate']); $discontinueYear = false; $discontinueMonth = false; if (!empty($packet['DiscDate'])) { list($discontinueMonth, $discontinueYear) = explode('/', $packet['DiscDate']); } unset($packet['IntroDate']); unset($packet['DiscDate']); $packet['introYear'] = $this->em->getReference( '\Stock\Entity\Year', $years[$introYear] ); $packet['introMonth'] = $introMonth; $packet['discYear'] = null; if ($discontinueYear) { $packet['discYear'] = $this->em->getReference( 'Stock\\Entity\\Year', $years[$discontinueYear] ); $packet['discMonth'] = $discontinueMonth; return $packet; } return $packet; } /** * @param array $packet * @param $modelId * @return array * @throws \Doctrine\ORM\ORMException */ public function prepareVehicleTypeDataForNewEntry(array $packet, $modelId) { $packet['VehicleType'] = $this->em->getReference( 'Stock\\Entity\\Category', $this->categoryMap[$packet['VehicleType']] ); $packet['Model'] = $this->em->getReference( '\Stock\\Entity\\Model', $modelId ); $packet['name'] = $packet['Variant']; $packet['createVersion'] = $this->store->version; $packet['updateVersion'] = $this->store->version; return $packet; } /** * @param array $packet * @param $type * @return bool */ public function vehicleTypeUpdateIsNeeded(array $packet, $type) { $needUpdate = false; foreach ($this->typeFieldsToMatch as $field) { if ($packet[$field] != $type->$field) { $needUpdate = true; $type->$field = $packet[$field]; } if (!$needUpdate) { if (false != $packet['discYear'] && is_null($type->discYear)) { $needUpdate = true; $type->discYear = $packet['discYear']; $type->discMonth = $packet['discMonth']; } } } return $needUpdate; } #-------------------------------------------------------------------------- SECTION /** * @return null */ public function processBatch() { usleep(500); $this->em->flush(); $this->em->commit(); $this->em->clear(); $this->em->getConnection()->close(); $this->em->beginTransaction(); $this->store->version++; $this->store->numItems = 0; } }