WordPress updates nginx and php-fm

I recently changed to php7 and some time later I realized updates is not working. Prompts for ftp permissions. This fixed it for me.

# diff /etc/php-fpm-7.0.d/www.conf /tmp/www.conf 
24c24
< user = nginx
---
> user = apache
26c26
< group = nginx
---
> group = apache

# service php-fpm restart
Stopping php-fpm-7.0:                                      [  OK  ]
Starting php-fpm-7.0:                                      [  OK  ]

Probably don’t need below if above is correct.

# tail -1 /sites1/blog/web/wp-config.php
define('FS_METHOD', 'direct');

Amazon SES Submission

I have experimented using the Amazon API to submit email to SES. As opposed to using SMTP the API may provide some advantages in some situations.

I jotted down some notes how I used the API and CommandPool since it took me a while to get it to work.

Out of scope:
1. SES and DKIM setup.
2. Requesting/Upping 1 email/sec SES rate limits.
3. SES sandbox and TO/FROM approvals.
4. Environment setup ie PHP 7/Nginx.
5. KEY/SECRET obfuscation from script ie store in hidden home folder.
6. API v1 and v2 versus v3 changes.

Input file so I can pass to CURL for testing.

$ cat emails.inp 
[
{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"},{"email":"success@simulator.amazonses.com"}
]

Run it from curl.

$ curl -H "Content-Type: application/json" -X POST --data-binary @emails.inp http://myserver.com/ses/amazon-commandpool_v2.php
About to send item 0 to: success@simulator.amazonses.com
About to send item 1 to: success@simulator.amazonses.com
About to send item 2 to: success@simulator.amazonses.com
About to send item 3 to: success@simulator.amazonses.com
About to send item 4 to: success@simulator.amazonses.com
Completed 0: 01000157c88196ba-e4a8d71a-c2a3-4d58-951a-e43639f29e05-000000 and result was: 200 
About to send item 5 to: success@simulator.amazonses.com
Completed 2: 01000157c88196b8-4b4bb9c4-a5af-4cd3-ad0c-808817910d12-000000 and result was: 200 
About to send item 6 to: success@simulator.amazonses.com
Completed 1: 01000157c88196b9-33fb8303-f7b5-451e-acd3-619b239053e6-000000 and result was: 200 
About to send item 7 to: success@simulator.amazonses.com
Completed 3: 01000157c88196b6-9a5d8224-441a-4f12-8b80-137716e7e11c-000000 and result was: 200 
About to send item 8 to: success@simulator.amazonses.com
Completed 4: 01000157c88196ba-c8cebab9-4397-490c-b425-10dcda4b99d4-000000 and result was: 200 
About to send item 9 to: success@simulator.amazonses.com
Completed 5: 01000157c881972b-3e243162-1eb4-4ad3-8c39-16ce31e10144-000000 and result was: 200 
Completed 8: 01000157c881975b-64270cf4-9666-408b-b16f-510a6cb4ff70-000000 and result was: 200 
Completed 7: 01000157c881974d-a2f67bfb-6200-44be-9164-16d099ca6dcf-000000 and result was: 200 
Completed 6: 01000157c8819734-8b3c6347-4a10-44b3-bd59-4e75fc7a2e9f-000000 and result was: 200 
Completed 9: 01000157c88197bc-574976c4-22e0-41ef-a5b6-1e810b666b58-000000 and result was: 200 

Total Execution Time: 0.44663500785828 Sec(s)

Script to accept emails in JSON and use the CommandPool API for asynchronous promises.

# cat amazon-commandpool_v2.php
<?php $time_start = microtime(true); if(strcasecmp($_SERVER['REQUEST_METHOD'], 'POST') != 0){ //throw new Exception('Request method must be POST!'); exit('Request method must be POST!'); } //Make sure that the content type of the POST request has been set to application/json $contentType = isset($_SERVER["CONTENT_TYPE"]) ? trim($_SERVER["CONTENT_TYPE"]) : ''; if(strcasecmp($contentType, 'application/json') != 0){ //throw new Exception('Content type must be: application/json'); exit('Content type must be: application/json'); } //Receive the RAW post data. $content = trim(file_get_contents("php://input")); //Attempt to decode the incoming RAW post data from JSON. $decoded = json_decode($content, true); //If json_decode failed, the JSON is invalid. if(!is_array($decoded)){ //throw new Exception('Received content contained invalid JSON!'); exit('Received content contained invalid JSON!'); } require '/sites1/myserver.com/web/ses/aws-autoloader.php'; use Aws\Exception\AwsException; use Aws\Ses\SesClient; use Aws\CommandPool; use Aws\CommandInterface; use Aws\ResultInterface; use GuzzleHttp\Promise\PromiseInterface; define('REGION','us-east-1'); $client = SesClient::factory([ 'version'=> 'latest',
    'region' => REGION,
    'credentials' => [
      'key'    => 'MYKEY',
      'secret' => 'MYSECRET',
    ]
]);

define('SENDER', 'myfromaddress@mydomain.com');
//define('RECIPIENT', array('success@simulator.amazonses.com','success@simulator.amazonses.com'));
//define('RECIPIENT', array('success@simulator.amazonses.com'));
define('SUBJECT','Amazon SES test (AWS SDK for PHP)');
define('BODY','This email was sent with Amazon SES using the AWS SDK for PHP.');

//$addresses = RECIPIENT;
$addresses = array_column($decoded, 'email');

$commandGenerator = function ($addresses) use ($client) {
    foreach ($addresses as $address) {
        // Yield a command that will be executed by the pool.
	$request = array();
	$request['Source'] = SENDER;
	$request['Destination']['ToAddresses'] = array($address);
	$request['Message']['Subject']['Data'] = SUBJECT;
	$request['Message']['Body']['Text']['Data'] = BODY;
        yield $client->getCommand('SendEmail', 
            $request
        );
    }
};

$commands = $commandGenerator($addresses);

$pool = new CommandPool($client, $commands, [
  'concurrency' => 5,
  'before' => function (CommandInterface $cmd, $iterKey) {
	$a = $cmd->toArray();
        echo "About to send item {$iterKey} to: " 
            . $a['Destination']['ToAddresses'][0] . "\n";
            //. print_r($cmd->toArray(), true) . "\n";
    },
  'fulfilled' => function (
        ResultInterface $result,
        $iterKey,
        PromiseInterface $aggregatePromise
    ) {
        echo "Completed {$iterKey}: {$result['MessageId']} and result was: {$result['@metadata']['statusCode']} \n";
        //echo "Completed {$iterKey}: {$result}\n";
    },
  'rejected' => function (
        AwsException $reason,
        $iterKey,
        PromiseInterface $aggregatePromise
    ) {
        echo "Failed {$iterKey}: {$reason}\n";
    },

]);

// Initiate the pool transfers
$promise = $pool->promise();

// Force the pool to complete synchronously
$promise->wait();

// Or you can chain then calls off of the pool
//$promise->then(function() { echo "Done\n" };

$time_end = microtime(true);
//dividing with 60 will give the execution time in minutes other wise seconds
//$execution_time = ($time_end - $time_start)/60;
$execution_time = ($time_end - $time_start);
//execution time of the script
echo "\nTotal Execution Time: ".$execution_time." Sec(s)\n";

Test from a web page as opposed to CURL command line.

# cat amazon-commandpool_v2_test.php
<?php //API Url $url = 'http://myserver.com/ses/amazon-commandpool_v2.php'; //Initiate cURL. $ch = curl_init($url); //The JSON data. $jsonData = array( array('email' => 'success@simulator.amazonses.com'),
    array('email' => 'success@simulator.amazonses.com'),
    array('email' => 'success@simulator.amazonses.com')
);
 
//Encode the array into JSON.
$jsonDataEncoded = json_encode($jsonData);
 
//Tell cURL that we want to send a POST request.
curl_setopt($ch, CURLOPT_POST, 1);
 
//Attach our encoded JSON string to the POST fields.
curl_setopt($ch, CURLOPT_POSTFIELDS, $jsonDataEncoded);
 
//Set the content type to application/json
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json')); 
 
//Execute the request
$result = curl_exec($ch);
?>

CORS Example with PHP

Mostly serving web pages you have the capability to do dynamic scripting since it is done on the same server. However if you do need to do this CORS allows the mechanism. Explained well at this link: http://enable-cors.org/

“JavaScript and the web programming has grown by leaps and bounds over the years, but the same-origin policy still remains. This prevents JavaScript from making requests across domain boundaries, and has spawned various hacks for making cross-domain requests.

CORS introduces a standard mechanism that can be used by all browsers for implementing cross-domain requests. The spec defines a set of headers that allow the browser and server to communicate about which requests are (and are not) allowed. CORS continues the spirit of the open web by bringing API access to all.”

In my case I wanted to proof this because I have a http enabled S3 bucket on Amazon AWS. Meaning all static. So one example of a need for this would be a contact form or quick lookup in a database etc. My example below is very simple. S3 website has a submit form button that use javascript to send the request to a PHP enable nginx server. PHP script look up the JSON value passed to it in an array and pass the value back.

S3 static web page:

<html>

<head>
<script type="text/javascript">

//window.onload = doAjax();

function doAjax(last_name) {
    var url         = "http://my.phpserver.domain/api/cors_contact.php";
    
    //var request     = JSON.stringify({searchterm:"doe"});
    //var search_st  = 'searchterm:' + 'last_name';
    //alert(search_st);
    var request     = JSON.stringify({searchterm:last_name});
    //alert (request);
    var xmlhttp     = new XMLHttpRequest();

    xmlhttp.open("POST", url);
    xmlhttp.setRequestHeader("Content-Type", "application/json; charset=UTF-8");
    xmlhttp.setRequestHeader("Access-Control-Allow-Origin", "*");
    xmlhttp.setRequestHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
    xmlhttp.setRequestHeader("Access-Control-Allow-Headers", "Content-Type");
    xmlhttp.setRequestHeader("Access-Control-Request-Headers", "X-Requested-With, accept, content-type");

    xmlhttp.onreadystatechange = function() {
        if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
            var jsondata = JSON.parse(xmlhttp.responseText);
            //document.getElementById("id01").innerHTML = xmlhttp.responseText;
            document.getElementById("id02").innerHTML = jsondata.word;
        }
    };

    xmlhttp.send(request);
}

function searchKeyPress(e)
{
    // look for window.event in case event isn't passed in
    e = e || window.event;
    if (e.keyCode == 13)
    {
        document.getElementById('Submit').click();
        return false;
    }
    return true;
}

</script>
</head>
<body>
<div id="id01"></div>
<div id="id02"></div>
<form name="contactform" id="controlsToInvoke" action="">
<table width="450px">
<tr>
<td valign="top"">
  <label for="last_name">Last Name *</label>
 </td>
<td valign="top">
  <input type="text" id="last_name" name="last_name" maxlength="50" size="30" onkeypress="return searchKeyPress(event);" />
 </td>
</tr>
<tr>
<td colspan="2" style="text-align:center">
  <input type="button" id="Submit" value="Submit" onclick="doAjax(document.getElementById('last_name').value)" />
 </td>
</tr>
</table>
</form>
</body>
</html>

PHP script.

In this case you can see all the headers are handled inside the PHP script. You can of course handle it with the web server also. Details for web servers here: http://enable-cors.org/server.html

<?php $dictionary = array('doe' => 'j.doe@email.domain', 'smith' => 'asmith@email2.domain', 'stranger' => 'noname@noname.com');

if ($_SERVER['REQUEST_METHOD'] == 'OPTIONS') {
    if (isset($_SERVER['HTTP_ACCESS_CONTROL_REQUEST_METHOD']) && $_SERVER['HTTP_ACCESS_CONTROL_REQUEST_METHOD'] == 'POST') {
        header('Access-Control-Allow-Origin: *');
        header('Access-Control-Allow-Headers: X-Requested-With, content-type, access-control-allow-origin, access-control-allow-methods, access-control-allow-headers');
    }
    exit;
}

$json = file_get_contents('php://input');
$obj = json_decode($json);

if (array_key_exists($obj->searchterm, $dictionary)) {
    $response = json_encode(array('result' => 1, 'word' => $dictionary[$obj->searchterm]));
}
else {
    $response = json_encode(array('result' => 0, 'word' => 'Not Found'));
}

header('Content-type: application/json');
header('Access-Control-Allow-Origin: *');
echo $response;

Thanks to this excellent reference. There are a lot of very confusing discussions on this topic. Especially around pre-flight checking.
http://www.mjhall.org/php-cross-origin-resource-sharing/

Creating a javascript array with one to many type relationship

Sometimes I need to build an array of lists with a one to many type relationship.

For example this data:
1. item1 has a parent of key1.
2. item2 has a parent of key2.
3. item3 has a parent of key1.

Using PHP after packing the array I will end up with something like this:

<?php

$m = array();

$m['key1'][] = 'item1';
$m['key2'][] = 'item2';
$m['key1'][] = 'item3';

print_r($m);

?>

Result:

Array
(
    [key1] => Array
        (
            [0] => item1
            [1] => item3
        )

    [key2] => Array
        (
            [0] => item2
        )

)

In javascript there is some options to do something similar. I ended up with something like this:
packedArr[‘key1’] = “item1,item3”
packedArr[‘key2’] = “item2”

Not exactly array in array[key] but the comma separated list did what I needed.

Here is the javascript itself. To test it you can use online javascript testers or just make up a html page with a button and maybe have onclick display document.getElementById(“demo”).innerHTML = output;

<script>
function packArr() {
var pA = Array();

var inA = [ 
  ["child1","parent1"],
  ["child2","parent2"],
  ["child3","parent1"],
  ["child4","parent1"],
  ["child5","parent2"]
  ];

var inALen = inA.length;
for (var i = 0; i < inALen; i++) {
    //alert(inA[i]);
  if (pA[inA[i][1]] == null) {
     pA[inA[i][1]] = inA[i][0];
  } else {
    pA[inA[i][1]] = pA[inA[i][1]] + "," + inA[i][0];
  }
}

res="";
for ( var key in pA ) { 
  res = res + key + ": " + pA[key] + "<br>";
};
//res = myArray["parent1"]; 
document.getElementById("demo").innerHTML = res;
}
</script>

Form Input Validation

Since I have to go research this every time I need it, I am jotting down a little how to. Frequently using html forms + PHP, I need to validate an input field.

Below is a recent simple javascript check for a regex style date + time input field. I also made this regex similar to msql timestamp for easier use in the database.

Of course there are a lot of ways to get this accomplished but my goal is fairly lightweight here. jquery and jquery date pickers are good options.

Like I said this is simple and use simple javascript alerts. You can make this a *lot* nicer with some ajax and css.

edit.php:
#########
Pay attention to the form name “frmEdit”, onSubmit action and the name of the field ie “dt_started”. Those are the ones that will get used in the javascript code.

<?php
?>
<html>
<head>
<link rel="stylesheet" type="text/css" media="screen" href="css/main.css" />
<script type="text/javascript" src="js/jquery-1.2.6.min.js"></script>
<script type="text/javascript" src="js/dateValidation.js"></script>
<title>Edit</title>
[...]
<form name="frmEdit" onReset="return confirm('Clear form?')" onSubmit="return ValidateForm()" action="<?php echo $_SERVER['PHP_SELF'] . "?edit=eTask&id=$id"; ?>" method="post">
<table border="0" width="100%">
[...]
<tr><td>dt_started: </td><td><input type="text" name="dt_started" maxlength=19 size=20 value="<?php echo htmlentities($details['dt_started']); ?>" />example: 2014-06-25 16:00:00 </td></tr>
<tr><td>dt_finshed: </td><td><input type="text" name="dt_finshed" maxlength=19 size=20 value="<?php echo htmlentities($details['dt_finshed']); ?>" />example: 2014-06-25 16:45:00 </td></tr>
[...]
<input type="reset" value="Reset" /> <input type="button" onClick="window.location='index.php'" value="Back" /></a></tr></td>
</table>
</form>

dateValidation.js:
##################

// Declaring valid date character, minimum year and maximum year
var dtCh= "-";
var minYear=1900;
var maxYear=2100;
[...]
function isDateTime(dtStr){
  var matches = dtStr.match(/^(\d{4})\-(\d{2})\-(\d{2}) (\d{2}):(\d{2}):(\d{2})$/);
  if (matches === null) {
    alert("Please enter a valid date")
    return false
  } else{
    var year = parseInt(matches[1], 10);
    var month = parseInt(matches[2], 10) - 1; // months are 0-11
    var day = parseInt(matches[3], 10);
    var hour = parseInt(matches[4], 10);
    var minute = parseInt(matches[5], 10);
    var second = parseInt(matches[6], 10);
    var date = new Date(year, month, day, hour, minute, second);
    if (date.getFullYear() !== year
      || date.getMonth() != month
      || date.getDate() !== day
      || date.getHours() !== hour
      || date.getMinutes() !== minute
      || date.getSeconds() !== second
    ) {
       alert("Please enter a valid date")
       return false
    } else {
       return true
    }
  }
}

function ValidateForm(){
  var dt=document.frmEdit.dt_started
  if (isDateTime(dt.value)==false){
    dt.focus()
    return false
  }
  var dt=document.frmEdit.dt_finshed
  if (isDateTime(dt.value)==false){
    dt.focus()
    return false
  }
return true
}

PHP SSH2 Bindings

PHP can use the bindings to the  libssh2 library which provide access to resources (shell, remote exec, tunneling, file transfer) on a remote machine using a secure cryptographic transport.   I documented some code snippets for future reference.

You can use ssh keys but for simplicity here making the call with a password:

$conn_id = ssh2_connect('server', 22);
ssh2_auth_password($conn_id, 'user', 'pwd');

echo("SOLARIS SERVER SNAPSHOTS - CREATION);
$stream=ssh2_exec($conn_id,"/usr/sbin/zfs list -t snapshot -o name,creation | grep $share | grep -v tank");
$outS="";
stream_set_blocking($stream, true);
while($o=fgets($stream)){
$outS = $outS . $o;
}
fclose($stream);
print_snaps_sorted($outS);

Slightly more complicated to make up the built-in javascript “command” you are passing when using a ZFS appliance. 

$chk_snaps_cmd="
run('select DBF');
run('snapshots');
snapshots = list();
for (i = 0; i < snapshots.length; i++) {
run('select ' + snapshots[i]);
creation = run('get creation');
printf('%s %s', snapshots[i],creation);
run('cd ..');
}";

$share = $_GET['share'];
$conn_id = ssh2_connect('server', 22);
ssh2_auth_password($conn_id, 'user', 'pwd');
echo('ZFS STORAGE APPLIANCE SNAPSHOTS - CREATION');
$stream=ssh2_exec($conn_id,"script run('shares select $share');" . $chk_snaps_cmd);
stream_set_blocking($stream, true);
while($o=fgets($stream)){
  $outS = $outS . $o;
}
fclose($stream);
print_snaps_sorted($outS);

Note:  I had an issues using ssh calls not returning results when using a non-privileged user whereas a root user worked.   Thanks Matt M for pointing out the obvious.  I needed to use the full path to the binary in this case /usr/sbin/zfs so most likely some path issue in the user shell.

Dynamic Proxy Auto Configuration

Some businesses still rely on the the Proxy auto-config file format originally designed by Netscape in 1996.  I have had instances where the javascript can get pretty complex and long.  Plus sometimes I have to rely on the javascript to report the IP address of the client accurately.  Sometimes the browser reports 127.0.0.1 or even IPv6 addresses.  Meaning the rules in the PAC file becomes useless since they are making decisions on where to direct the client based on its subnet.

One other idea is to use PHP to generate the javascript dynamically.  For one you can really simplify the code by reading your lists from files into arrays dynamically but also you can correctly capture the IP address of the client.

Example: I simplified it a little for readability and its been a while since I played with this so don’t recall if this one actually was a working copy.

<?php
    $proxy = "mainsquid001.domain.com";
    $port  = "3128";

    header ("Content-type: application/x-ns-proxy-autoconfig");
    header ("Date: " . gmdate('D, d M Y H:i:s \G\M\T', time ()));
    header ("Last-Modified: " . gmdate('D, d M Y H:i:s \G\M\T', time ()));
    header ("Expires: " . gmdate('D, d M Y H:i:s \G\M\T', time () + 60 * 30));

    #echo "// Request from: " . $_SERVER ['REMOTE_ADDR'] . "\n";
    #echo "// OS: " . $_SERVER ['HTTP_USER_AGENT'] . "\n";

    $netProxyMap = array('172.16'=>'uksquid001','172.19'=>'casquid001');
    // I removed a lot of subnets from above array for readability.
    $ipA = explode(".", $_SERVER ['REMOTE_ADDR']);
    $bNetPart = $ipA[0] . "." . $ipA[1];
    $cNetPart = $bNetPart . "." . $ipA[2];
  
    if ( array_key_exists($cNetPart, $netProxyMap) ) {
      $proxy = $netProxyMap[$cNetPart] . ".domain.com";      	
    } else {
	if ( array_key_exists($bNetPart, $netProxyMap) ) {
          $proxy = $netProxyMap[$bNetPart] . ".domain.com";      	
        }
    }
?>

// Proxy Servers
var proxy = "PROXY <?php echo $proxy; ?>:<?php echo $port?>;";
var dmzproxy = "PROXY dmzproxy.domain.com:3128;";

// Proxy Logic
function FindProxyForURL(url, host)
{
    if (url.substring(0,6)=="https:") { return "DIRECT"; }

    else if (shExpMatch(url,"*.google.com*") ||
      shExpMatch(url,"*.gotomeeting.com*") 
      { return "DIRECT"; }

    else if ((host == "localhost") || (shExpMatch(host, "localhost.*")) || (host == "127.0.0.1")) 
     { return "DIRECT;"; }

    else if ( (host == "hostedext1.domain.com") || (host == "hostedext2.domain.com") 
     { return olddmzproxy; }

    else if ((host == "www.domain.com") || (host == "dmzapp1.domain.com")  
     { return dmzproxy; }

    else if (shExpMatch(url,"http://domain.com/")) { return dmzproxy; }

    else if (dnsDomainIs(host, ".domain.com")) { return "DIRECT;"; }

    else if (isPlainHostName(host)) { return "DIRECT;"; } 

    else return proxy;
}