toolx
Framework
Documentation for Developers
The brand new platform to manage OLX’s internal processes
Hello !
1. Introduction
ToOLX is a modular MVC Framework, developed from scratch and designed to provide the resources needed (Models, Libraries, ACL, etc) allowing to build robust and agnostics web applications. Giving each application/module autonomy and ability to interact with the major components/services of OLX.
2. Value + Mission
Under this paradigm, already have been written several internal OLX applications, among which we highlight: "Category Tree Manager", "Keywords Report of Ads" (Product Department), "Deploy Seo Metatags" (SEO Department), "New Translation Tool" (Trans Department), "Expiration rules", "Item rebump", "Campaign URL shortening" (marketing).
The main philosophy of ToOLX is to focus on the simplicity and scalability. Allowing developing modules in record time with features that provide value and own high user experience.
3. Ready to Dev
The purpose of this documentation is that you can understand the high-level backbone of the Framework and be ready to develop.
How to Install ?
1. Clone code and configs repos
# Clone Toolx-Main repo
~ git clone git@github.com:olx-inc/toolx.git
# At the same level than Toolx-Main repo, clone Toolx-Conf repo
~ git clone git@github.com:olx-inc/toolx-conf.git
# Enter CHROOT
# Go to /home/httpd/sites/toolx/app and run
~ ln -s /home/httpd/sites/toolx-conf Configs
2. Configure Apache web server
# Create VirtualHost
<VirtualHost *:80>
ServerName toolx.olx.com
DocumentRoot "/home/httpd/sites/toolx"
ServerAlias toolx.olx.com
<Directory "/home/httpd/sites/toolx">
Options -Multiviews
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ start.php [L]
</Directory>
</VirtualHost>
3. Create aditional folders
# Go to Toolx base path
~ cd /home/httpd/sites/toolx
# Create "files" and "cache" folders with read/write privileges
~ mkdir -m 777 files
~ mkdir -m 777 cache
4. Set database credentials
# Open and edit db.yml config file
~ vim app/Configs/db.yml
# Replaces 'secret' value of key password with the original password for each enviroment and save changes.
5. Add host for ABL Service
# Add IP-Mapping for ABl service in your local hosts file.
166.78.73.43 abl-server.olx.com.ar
# For more information about the OLX internal services consumed by ToOLX, see the follow config file:
~ cat app/Configs/services.yml
6. Go to web browser, enter: http://toolx.olx.com and Rocks!!
Modules
1. A module is an App
The modules in ToOLX represent applications, each of them is different and totally independent. The modules have their own MVC architecture (models, views, controllers) and the ability to implement their own configuration files and libraries/helpers.
The Framework will be in charge through the ACL to determine that each controller actions each user can access depending on their level of permissions and roles. This will be explained later in the ACL section.
2. How to create a new Module ?
Create a new module is quite easy. ToOLX provides a toolkit that runs from the terminal and automates this process.
# Go to ToOLX /app folder
~ cd /toolx/app
# In app folder, execute in a command line the toolkit/generate
~ ./toolkit/generate
# A continuacion el prompt solicitara ingresar el nombre del Modulo
Welcome to module generator
Please enter the namespace ( only first uppercase ):
[Yourmodulename]
# Finalmente vera un mensaje de confirmacion y el modulo se habra creado.
Module [Yourmodulename] created successfully ;)
3. Basic module structure
~/toolx/app/Modules/Test
├── Configs
├── Controllers
│ └── DefaultController.php
├── Models
├── Public
└── Views
├── Default
│ └── Default.twig
└── menu.twig
6 directories, 3 files
3.2. Are you curious? Advanced module structure
~/www/toolx/app/Modules/Metatag
├── Configs
│ ├── areas.yml
│ └── placeholders.yml
├── Controllers
│ ├── ChangelistController.php
│ └── ManagerController.php
├── Library
│ └── Parameters.php
├── Models
│ ├── Action.php
│ ├── Log.php
│ ├── Metatag.php
│ └── Package.php
├── Public
│ ├── css
│ │ ├── changelist.css
│ │ └── manager.css
│ ├── js
│ │ ├── changelist.js
│ │ ├── jquery.editableGrid.js
│ │ ├── jquery.editableGrid.min.js
│ │ ├── jquery.textcomplete.js
│ │ ├── jquery.textcomplete.min.js
│ │ ├── main.js
│ │ ├── pack.js
│ │ └── pack.wizard.js
│ └── less
│ ├── changelist.less
│ └── manager.less
└── Views
├── Changelist
│ └── List.twig
├── Manager
│ ├── Manager.twig
│ └── Templates
│ ├── AddForm.twig
│ ├── ApplyModal.twig
│ ├── Package.twig
│ └── SearchForm.twig
└── menu.twig
12 directories, 28 files
Controllers
1. Description of Controller
The Controller is the entry point for our module. For a Controller to be visible by the "Framework" the filename and the class should be suffixed with the word "Controller". The Controller has "Actions", this will be our point of entry when build our module paths. The controllers must be extended from: \Framework\BaseController
2. Actions
2.1 Description of Actions
The Actions of a Controller are methods that take care of preparing the information before being returned to the view, or just to send some kind of response: eg: JSON. The Action is very important because it is also used by the Framework to build the "URI" from our module in order to generate the routing of the application. The method name must end with the word "Action" and must be a public method. Just as the name implies will define the action of the MVC routing.
2.2 Routing to an Action
The Routing is composed as follows, module/controller/action For example if we want to access the method listingAction () that is inside DemoController in the module named Test, the URI to this Action would be test/demo/listing. The keywords: Controller and Action are omitted from the URI. In case of not specify the Controller and Action, then the Framework by default will look for the Controller: DefaultController and Action: DefaultAction under that concept, if we invoke the following URI: /test the routing of the Framework will search for Test/DefaultController/defaultAction.
3. Controllers Methods
Here we will see a number of functions that allow us to take full advantage of our Controller.
/* Render Method */
/* This method takes care of sending the data to the view, to then be displayed on the page.*/
$this->_render($view, $params = array(), $contentType = 'text/html');
// view -> path *.twig we want to render (default route module/controller/ will be used to complete the path *.twig if the full path is not specified).
// params(Optional) -> Array with the data sent to the view.
// contentType(Optional) -> Content type of the view.
/* JSON Response */
/* This method renders the data as a JSON with the corresponding content type. */
$this->_jsonResponse($data);
// data -> Data to be converted into JSON. Ex: (Array)
/* CSV Response */
/* This method only adds the appropriate headers to return a CSV response. */
$this->_csvResponse($filename);
// filename -> Filename to be sent in the response header.
3.1 Overwrite Methods
/* userAllowed() */
/* This function is used by the "ACL" to determine whether a user has access to the controller or not. To allow any user access without the need for permissions it should override it just returning true. */
function userAllowed()
{
return true;
}
/* requireLogin */
/* This variable determines whether it is necessary that the user is logged in or not to display the controller/action. If it is overwritten with false, then it will not require the user to be logged in to view that action. */
protected $_requireLogin = false;
3.2 simple Controller example
namespace Test\Controllers;
class DefaultController extends \Framework\BaseController
{
public function defaultAction()
{
$parameters = array('foo' => 'var');
$this->_render('Default.twig', $parameters);
}
}
4. AJAX in Controllers
AJAX calls usually do not require special permissions to access them for this very reason we must generate a unique crontroller for them which must extend from: \Framework\AjaxController
4.1 simple AJAX Controller example
namespace Test\Controllers;
class AjaxController extends \Framework\AjaxController
{
public function getDateAction()
{
$date = date('Y-m-d');
$this->_jsonResponse($date);
}
}
Helpers
The Helpers as the name suggests, are small functions that help us to code in a controller
1. Config
use Framework\Config;
Config::get($pathToConfig, $default);
//Returns the settings saved in the *.yml files that are housed in the "Configs" folder where the module is being executed, and those that are inside the configs of the framework, the second parameter is the value to be returned if the function can not find the requested resource, in case of not assign any value, it will return "null" by default.
use Framework\Config;
Config::get('fileConfig.config.value', false);
//This function will look for the "fileConfig.yml" file in the "Configs" folder in the "module" and "framework", if found will return the value of "value" in the key "config", if not found, will return the specified value, in this case "false".
2. Request
use Framework\Request;
//This Helper has many useful methods, the most used are:
Request::getAll();
//Returns all the values of $ _POST, $ _GET and $ _FILE in a single array.
Request::get('valueName', $defaultReturn);
//Returns the requested value and absence returns the value specified in $defaultReturn.
Request::isMethod('METHOD');
//Returns true/false depending on which method was used, the values for "METHOD" may be for example: "POST", "GET", "PUT", "HEAD", etc.
Request::isAjax();
//returns true/false if the method used was AJAX.
Request::getComponents();
//Returns an object with the current values of module/controller/action.
3. Session
use Framework\Session;
Session::set($key, $value);
//Saves in the user session, the value of $value on the $key requested.
Session::get($key);
//Returns the $key required from the user session.
Session::delete($key);
//Delete from the user session, the requested $key.
4. Utils
use Framework\Utils;
//this Helper has many useful methods, the most used are:
Utils::olxSlug($string);
//returns a string in slug format, used by OLX, by cleaning special characters, and separate them by "-";
Utils::getUploadPath();
//returns a string with the full path to the directory "/files" of the framework.
Utils::dump($data, $hidden = false);
//prints on screen a dump from the variable $data, it can be hidden in the view as a comment if true is passed as the second parameter.
5. Exceptions
use Framework\HttpException;
throw new HttpException($message, $code);
//This Helper is a shortcut to generate exceptions 404, 500 among others.
//use examples:
throw new HttpException("page not found", 404);
throw new HttpException("internal error", 500);
//This will render a stylized page with the corresponding error and message.
Libraries
1. JIRA
use Framework\Library\Jira;
$jira = new Jira();
//This library is quite extensive, below the most basic options will be displayed.
$jira->createIssue($projectKey, $summary, $issueType, $options = array());
// projectKey -> is the value of the key that was used in the issue, for example 'TOOLX', 'FP', 'FORUM'.
// summary -> is a string with the description of the issue.
// issueType -> it is the type of issue generated, for example 'Project', 'Task'.
// options(optional) -> depending on the key used and the type of project this array will have different structure, one must check which are mandatory for each case.
//This function returns an array with the response of the query, where response ['key'] is equal to the key of the issue generated, for example 'TOOLX-128', 'FP-64'.
$jira->getIssue($issue);
// issue -> equals the requested key issue, for example 'TOOLX-128 "," FP-64'.
//This function returns an array with all the data for that issue.
$jira->createAttachment($issue, $filename, $filePath, $options = array());
// issue -> equals the requested key issue, for example 'TOOLX-128 "," FP-64'.
// filename -> is a string with the name you want to assign to that file.
// filePath -> full path of the file on the server.
// options(optional) -> depending on the key used and the type of project this array will have different structure, one must check which are mandatory for each case.
2. ABL
use Framework\Library\AblRestProvider;
//AblRestProvider it is the library to connect to ABL
$abl = new AblRestProvider($env);
// env -> is equal to the environment which you want to connect with ABL: prod, dev, etc.
$abl->addParam('tr')
->addParam('location')
->addParam('country')
->addParam(2);
//those parameters are added to connect the specific resource that is required, in this case "location".
$response = $abl->getResponse();
//the query is executed with getResponse(), which returns an array with the response from ABL.
Location Resource
//Get list of Countries visible by the User logged
$countries = \Framework\Session::get('user')->getCountries();
//Get full list of Countries (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$countries = $locationModel->getCountries();
//Get REDUCED Country info by id (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$country = $locationModel->getCountry($countryId);
//Get FULL Country info by id (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$country = $locationModel->getCountryFull($countryId);
//Get list of Countries by Markets ids (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$countries = $locationModel->getCountriesByMarkets($marketsIds);
//Get list of States by Country id (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$states = $locationModel->getStates($countryId);
//Get list of Cities by Country id and State id (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$cities = $locationModel->getCities($countryId, $stateId);
//Get list of Languages by Country id (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$languages = $locationModel->getLanguages($countryId);
//Get list of Categories by Country id (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$categories = $locationModel->getCategories($countryId);
//Get FULL info of a L1 (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$l1 = $locationModel->findL1($countryId, $categoryId);
//Get FULL info of a L2 (retrieved from ABL)
$locationModel = new \Framework\Models\Location();
$l2 = $locationModel->findL2($countryId, $categoryId, $subcategoryId);
User Resources
1. User
//Get User business object from the current logged in user.
$user = \Framework\Session::get('user');
2. Roles
//Get Role id of the User logged in
$id = \Framework\Session::get('user')->getRoleId();
//Get Role name of the User logged
$name = \Framework\Session::get('user')->getRoleName();
//Get Roles tree of the User logged
$tree = \Framework\Session::get('user')->getRolesTree();
//Get list of Roles id extracted from the Roles tree of the User logged
$ids = \Framework\Session::get('user')->getMyRoles();
3. Markets
//Get list from the User logged
$markets = \Framework\Session::get('user')->getMarkets();
//Get name by id
$marketModel = new \Framework\Models\Market();
$name = $marketModel->findById($marketId);
//Get market info (id, name, rtl) by key (retrieved in ABL's country resource)
$marketModel = new \Framework\Models\Market();
$market = $marketModel->find($marketKey);
//Know whether or not the User logged has all the markets
$hasAllMarkets = \Framework\Session::get('user')->hasAllMarkets();
Logs
To log in ToOLX you need to call the native syslog() in PHP, but with some details to consider, the first parameter is the constant LOG_PHP as second parameter the message to be logged as a prefix should always put "ToOlx-App-ModuleName:" where "ModuleName" is the name of the module in which we are running the log.
//info log example.
syslog(LOG_INFO, 'ToOlx-App-Smartling: This is an Information log');
//error log example.
syslog(LOG_ERR, 'ToOlx-App-Rebump: This is an Error log');
Crons
A Cron in ToOLX is as easy as writing a class extending from "\Framework\BaseCron" in which we rewrite the method "_Execute()". To have a global view on the execution of it, use the method "writeOutput()" to generate a log with the current process of the Cron.
//simple cron example
//this class must be in the module that we are working at, inside of the "Crons" folder
namespace Test\Crons;
class TestCron extends \Framework\BaseCron
{
protected function _execute()
{
$this->writeOutput("generate an array of 5 values");
$vars = array(1,2,3,4,5);
$this->writeOutput("start iteration of those values");
foreach ($vars as $var) {
$this->writeOutput("iterate value $var");
$this->writeOutput("waiting for 2 seconds");
sleep(2);
}
$this->writeOutput("end of iteration");
}
}
In order to run the Cron we must create a php file in the folder "/batch" to execute it.
//cron entry point example "/batch/test.php"
require_once 'bootstrap.php';
$cron = new \Test\Crons\TestCron();
$cron->run();
For example to run this "Cron", From the root of ToOLX we should run the command: "php batch/test.php" to view a verbose version of this Cron, we must pass the parameter "debug" to the Cron eg: "php batch/test.php debug". This will print the log on screen in real time.