addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1linklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Milwaukee PHP Users Group Message Board MKEPUG Discussion Forum › Noob in need of some guidance?

Noob in need of some guidance?

whispers007
whispers007
Oak Creek, WI
Post #: 2
Hi group..

(php noob here.. so be nice!) ;)

I am trying to work my way through a 'simple' script.. to manipulate/search/create some text files...

I have done similar 'small' scripts like this before... but now I am working my through it and coming up with errors.

ie: "Fatal error: Maximum function nesting level of '100' reached, aborting! in C:\wamp\www\ahc_guarSearch\guarMatch.php­ on line 47"

so Im obviously doing somethign wrong.. probably my logic/approach to it..



summary/goal:

I need to take a file (scrape directory for file name).. that has lines of TEXT with delimited data using the "^" character...

I push each 'line' into an array........

(no telling how many lines in total... a couple to a few thousand)

this NEW array on each line in the .txt file, is really my 'source' array..... I need ot take the first array index [0] in each line.. and 'search through a directory of .txt files in a folder/directory..


look in EACH of these .txt file the value in lineArray[0]... and if a match.. grab a piece of data in the newly searched in file.


kinda like a search tool of sorts..


I have a TEXT file of lines/data...

when parsed into an array.. the [0] index in each line is the search term used to recursively search though the target directory of multiple .txt files.... if a match is found.. it takes a piece of the datafrom the line where the match was found.. and appends it to the original array.


steps I have done/tried so far:

1.) scraped direct for my 'main/source' file
2.) added file names to an array to loop through (most likely, there will only be one 'source' file..but wanted to keep things dynamic a bit for future uses.
3.) parse each text file in the array above adding each line to another array

source data looks like:
108383442^$14.51^$3.63^25.00%^$10.88^P­AY
106620416^$0.55^$0.14^25.00%^$0.41^PAY­

4.) I then scrape the 'target' (search) directory for a list of files names in it.. and add this to an array.


5.) Here is where Im having trouble.....

I then need to:

a.) use my 'source array' (has the search term in each line at index[0]) ...grab the value at sourceArray[x][0]
b.) take this value and check/search the first file in the 'searchArray' file names... and see if there is a match in any of the lines....

if "NOT".... then I need to keep same search term and move onto the next searchable file/name

if "YES"... then I need to grab a piece of data from the line where a match was found...


I hope this makes sense?? Im getting a bit caught up on all the lopping/nesting/looping..etc..

the searchable text files has data like this:


01^000103408883^0000850094^Doe,Carole,­K^293348768^19661118^Female^Single


And to show I gave it a shot first before asking for help.. here is my current script:


anybody have some direction for me? What Im doing wrong? better way? (trying not to get too fancy.. as I want to understand the script too)

thanks guys..
whispers007
whispers007
Oak Creek, WI
Post #: 3
code:


<?php

$reportName = "guarMatch_";

//array of all HAR files in directory needing to be scraped
$harFileArray = array();
$totalHarFiles = 0;

//array of all har records found during har scrape
$harRecordArray = array();
$totalHarRecords = 0;


//array of all SEARCH/PLACEMENT files in directory needing to be searched
$searchFileArray = array();
$totalSearchFiles = 0;

//array of all records found during the file(s) search
$searchMatchArray = array();
$totalMatchRecords = 0;


$currentHarFile = 0;
$currentSearchFile = 0;
$tempCount = 0;
$lineArray = array();
$detailArray = array();
$fileName = "";

function searchContents(){
global $searchFileArray;
global $currentSearchFile;
global $harRecordArray;
global $totalSearchFiles;
global $tempCount;
global $searchMatchArray;
global $totalMatchRecords;

$fp = fopen('placementDB/' . $searchFileArray[$currentSearchFile],'rb­');

while(!feof($fp)) {
$newLine = fgets($fp);
if($newLine === false) {
//end of file, no more lines
//echo ("NO MATCH FOUND END OF FILE. <br>");
$currentSearchFile++;
break;
}else{
// ... do something with $newLine
//search $newLine string for search word
if (strpos($newLine, $harRecordArray[$tempCount][0])) {
echo ('<br> MATCH FOUND - <br>');
//array_push($searchMatchArray, ($newLine . "^" . $harRecordArray[$tempCount][0]));

// ... do something with $newLine
$lineArray = explode("^", $newLine);

array_push($searchMatchArray, $lineArray[2]);

print_r("GUAR#: " . $lineArray[2] . "<br>");
print_r("HAR: " . $harRecordArray[$tempCount][0]);

$totalMatchRecords++;
$tempCount++;
break;
}else{
//no match found this line.., go onto next line
//$currentSearchFile++;
}
}
}
//close file
fclose($fp);

if($currentSearchFile < $totalSearchFiles){
searchContents();
}else{
//do nothing..last file searched
print_r("<br> DONE SEARCHING ALL FILES!");
print_r("<br> FINAL TOTAL (Array Count): " . count($recordArray) . "<br><br>");
//print_r($recordArray);
//exportData();
}
}


function getFileNames(){

global $searchFileArray;
global $totalSearchFiles;
global $searchRecordArray;
global $totalSearchRecords;

if ($handle = opendir('placementDB/')) {
while (false !== ($searchFile = readdir($handle))) {
if($searchFile !='.' && $searchFile !='..'){
array_push($searchFileArray, $searchFile);
$totalSearchFiles = count($searchFileArray);
}
}
closedir($handle);

//check array contents
//print_r($searchFileArray);
print_r("<br><br>Total FILES TO SEARCH: " . $totalSearchFiles ."<br><br>");
}
//call next function to search file contents
searchContents();
}


function getHarIDs(){

global $harFileArray;
global $totalHarFiles;
global $harRecordArray;
global $totalHarRecords;
global $currentHarFile;
global $lineArray;

if ($handle = opendir('.')) {
while (false !== ($harFile = readdir($handle))) {
if($harFile != 'og' && $harFile != 'og/' && $harFile != 'exported' && $harFile != 'exported/' && $harFile != 'placementDB' && $harFile !='guarMatch.php' && $harFile !='.' && $harFile !='..'){
array_push($harFileArray, $harFile);
$totalHarFiles = count($harFileArray);
}
}
closedir($handle);

//check array contents
print_r($harFileArray);
print_r("<br><br>Total FILES TO GRAB HAR ID's FROM: " . $totalHarFiles ."<br><br>");
}

//------------------ parse file array and get HAR ID & populate array ----------------------//

$fp = fopen($harFileArray[$currentHarFile],'rb­');
$tempCount = 0;

while(!feof($fp)) {
$newLine = fgets($fp);
if($newLine === false) {
break;
}else{
// ... do something with $newLine
$lineArray = explode("^", $newLine);

//$lineArray[0] = HAR ID in file
array_push($harRecordArray, $lineArray);
print_r($lineArray[0] . "<br>");
//print_r($harRecordArray[$tempCount][0]­ . "<br>");

$totalHarRecords++;
$tempCount++;
}
}
print_r("TOAL HARS FOUND IN ALL FILES: ". $totalHarRecords . "<br>");
//$currentHarFile++;
//close file
fclose($fp);
getFileNames();
}
getHarIDs();

?>


thanks!


update: I see this is due to an PHP.ini setting dealign with Zend? or Xdebug?

I edited .ini file.. restarted services.. and the error went away..

thanks!
whispers007
whispers007
Oak Creek, WI
Post #: 4
anyone around? :)

this is the current state of the script:


<?php

$reportName = "guarMatch_";

//array of all HAR files in directory needing to be scraped
$harFileArray = array();
$totalHarFiles = 0;

//array of all har records found during har scrape
$harRecordArray = array();
$totalHarRecords = 0;


//array of all SEARCH/PLACEMENT files in directory needing to be searched
$searchFileArray = array();
$totalSearchFiles = 0;

//array of all records found during the file(s) search
$searchMatchArray = array();
$totalMatchRecords = 0;


$currentHarFile = 0;
$currentSearchFile = 0;

$tempCount = 0;
$lineArray = array();
$detailArray = array();
$fileName = "";



function exportData(){

//where I'll loop through my array(s) and create a.txt file

}


function searchContents(){
global $searchFileArray;
global $currentSearchFile;
global $harRecordArray;
global $totalSearchFiles;
global $tempCount;
global $searchMatchArray;
global $totalMatchRecords;
global $totalHarRecords;

$fp = fopen('placementDB/' . $searchFileArray[$currentSearchFile], 'rb');

while(!feof($fp)) {
$newLine = fgets($fp);

if($newLine === false) {
//end of file, no more lines
echo('File #: ' . $currentSearchFile . ' ');
echo ("NO MATCH FOUND / END OF FILE. <br>");

if($currentSearchFile < $totalSearchFiles){
$currentSearchFile++;
//close file
fclose($fp);
echo('More files left to search.....<br><br>');
searchContents();
}

}else{
// ... do something with $newLine
//search $newLine string for search word
if (strpos($newLine, $harRecordArray[$tempCount][0])) {
echo ('<br> MATCH FOUND - <br>');
//array_push($searchMatchArray, ($newLine . "^" . $harRecordArray[$tempCount][0]));

// ... do something with $newLine
$lineArray = explode("^", $newLine);

array_push($searchMatchArray, $lineArray[2]);

print_r("GUAR#: " . $lineArray[2] . "<br>");
print_r("HAR: " . $harRecordArray[$tempCount][0] . "<br><br>");

$totalMatchRecords++;
$tempCount++;

//close file
fclose($fp);

//check to see if all hars found a match
if($totalMatchRecords < $totalHarRecords){
//break;
$currentSearchFile = 0;
searchContents();
}else{
//no match found this line.., go onto next line
//$currentSearchFile++;
break;

}
}
}
}

/*
if($currentSearchFile <= $totalSearchFiles){
$currentSearchFile++;
//close file
fclose($fp);
searchContents();
}else{
//do nothing..last file searched
print_r("<br> DONE SEARCHING ALL FILES!");
print_r("<br> FINAL TOTAL (Array Count): " . count($searchMatchArray) . "<br><br>");
//print_r($recordArray);
//exportData();
break;
}

//check to see if all hars found a match
if($totalMatchRecords < $totalHarRecords){
searchContents();
}else{
//no match found this line.., go onto next line
//$currentSearchFile++;
break;
}
*/

}


function getFileNames(){

global $searchFileArray;
global $totalSearchFiles;
global $searchRecordArray;
global $totalSearchRecords;

if ($handle = opendir('placementDB/')) {
while (false !== ($searchFile = readdir($handle))) {
if($searchFile !='.' && $searchFile !='..'){
array_push($searchFileArray, $searchFile);
$totalSearchFiles = count($searchFileArray);
}
}
closedir($handle);

//check array contents
//print_r($searchFileArray);
print_r("<br><br>Total FILES TO SEARCH: " . $totalSearchFiles ."<br><br>");
}
//call next function to search file contents
searchContents();
}


function getHarIDs(){

global $harFileArray;
global $totalHarFiles;
global $harRecordArray;
global $totalHarRecords;
global $currentHarFile;
global $lineArray;

if ($handle = opendir('.')) {
while (false !== ($harFile = readdir($handle))) {
if($harFile != 'og' && $harFile != 'og/' && $harFile != 'exported' && $harFile != 'exported/' && $harFile != 'placementDB' && $harFile !='guarMatch.php' && $harFile !='.' && $harFile !='..'){
array_push($harFileArray, $harFile);
$totalHarFiles = count($harFileArray);
}
}
closedir($handle);

//check array contents
print_r($harFileArray);
print_r("<br><br>Total FILES TO GRAB HAR ID's FROM: " . $totalHarFiles ."<br><br>");
}

//------------------ parse file array and get HAR ID & populate array ----------------------//

$fp = fopen($harFileArray[$currentHarFile],'rb­');
$tempCount = 0;

while(!feof($fp)) {
$newLine = fgets($fp);
if($newLine === false) {
break;
}else{
// ... do something with $newLine
$lineArray = explode("^", $newLine);

//$lineArray[0] = HAR ID in file
array_push($harRecordArray, $lineArray);
//print_r($lineArray[0] . "<br>");
//print_r($harRecordArray[$tempCount][0]­ . "<br>");

$totalHarRecords++;
$tempCount++;
}
}
print_r("TOAL HARS FOUND IN ALL FILES: ". $totalHarRecords . "<br>");
//$currentHarFile++;
//close file
fclose($fp);
getFileNames();
}
getHarIDs();

?>




I am now getting this error:


Fatal error: Maximum execution time of 30 seconds exceeded in C:\wamp\www\ahc_guarSearch\guarMatch.php­ on line 52

which happens to be this line:


while(!feof($fp)) {



what am I doing wrong here? Is this just not the right approach to this?


Aaron S.
aaronsaray
Milwaukee, WI
Post #: 95
Hello and welcome. Sorry I don't have much time to check through the code... but here are my thoughts.

Chances are from what you're describing, the code is inefficient - it shouldn't be taking that long to run. However, saying that - and having no solutions - isn't that helpful is it? :) Perhaps someone else has a chance to help.

However, I can point you to set_time_limit PHP function - so you can increase the timeout on your script. Best of luck.
whispers007
whispers007
Oak Creek, WI
Post #: 5
Thanks for the reply... :)

Question:

Would it maybe be ‘better’ to write some php script to take this directory of 200+ (and growing) .txt files and parse them into a database? (MySQL) FIRST???

To get the bulk of these done/into a database.. and going forward.. just add the new .txt file to the DB as they come?

And then have another php script that sorta does the same original goal.. but instead of parsing/opening 200+ text files each time.. it uses the database?


anyways.. at least Im not having any 'script/code/syntax' errors..LOL.. Just limit/max errors.. (going through the .ini file now)


Beth Tucker L.
e3betht
Verona, WI
Post #: 33
A database may help you, and it would be pretty easy to import the data since it is already delimited. Is there a reason the data is currently in text files (efficiency, etc.)? Also, databases are not always more efficient, but they do make searching data a little more simple to code. You could also look at using a text search program so scan and index the files, or even use the OS to search. Is there a reason to read everything into arrays first? Or could you just use a text search?
whispers007
whispers007
Oak Creek, WI
Post #: 6
hi..

thanks for the reply..

hopefully I can answer some questions..

1.) its in text files.. because thats how they come/we get them.... (this is more of a retro-active fix for a co-worker now to deal with importing the old 'stock' of text files into the DB.. going forward.. will run this 'import/parse' script each each on the new file


I dont think a regular text search will work from an OS.. (at least that "I" know how to do) haha..


the 'searching' part would be a MASS group of .txt files (php or Zend/Xdebug..etc seems to crap out.. disabling that still timed out got errors).. so dumping to a database seems the best/only approach left. :)


I believe this project has evolved into TWO parts now..


1.) creating a script to take this inventory of .txt files.. and parse/dump them to a database..

(working on that now.. but am back to ask for some more direction/help) :)





2.) searching..

this is NOT a 1 term 'search and results'..

it is a TEXT file with XXXX lines in it.. (all delimited)

I need to explode/parse each line.. grab the value at index[x]... and create a LIST of all these value (for as many lines are in the text file).. and use this as my SEARCH LIST...

once a MATCH is found in the database.. I grab a piece of data associated with that matched term/# form the database... and append it to the original 'line array' I got my original search term from (index[x])

Hopefully that made sense? LOL



Ayways.. moving forward.. I am currently on the dumping to the database portion of the script.. but it is NOT dumping any records to the DB?

I am not even getting any errors? So Im not sure where to trouble shoot??


her is current script:

everything 'seems' to be working.. and in these 'functions' where I dump to the DB.. my echo statemts print out fine.. and the data/values are there.. but no DB adding or errors?

php:

function insertRecords(){

global $allRecords, $myDB, $subDate, $allLinesArray;
//echo('INSERTING NEW RECORD<br><br>');
//echo('TOTAL RECORDS: ' . count($paymentAmount) .'<br>');

for($i=0; $i < count($allLinesArray); $i++) {


if($i){
$allRecords .= ',';
}
//these work fine?
//echo("CHECK: " . $allLinesArray[$i][142] . "<br>");
//echo("CHECK: " . $allLinesArray[$i][2] . "<br>");
//echo("looping: " . $i);

//** first index is ID# (auto-populated)
$allRecords .= "('','$allLinesArray[$i][1]','$allL­inesArray[$i][2]','$allLinesArray[$i][3]­','','','$allLinesArray[$i][4]','$allLin­esArray[$i][5]','$allLinesArray[$i][6]',­'$allLinesArray[$i][7]','$allLinesArray[­$i][8]','$allLinesArray[$i][9]','$allLin­esArray[$i][10]','$allLinesArray[$i][11]­','$allLinesArray[$i][12]','$allLinesArr­ay[$i][13]','$allLinesArray[$i][14]','$a­llLinesArray[$i][15]','$allLinesArray[$i­][16]','$allLinesArray[$i][17]','$allLin­esArray[$i][18]','$allLinesArray[$i][19]­','$allLinesArray[$i][20]','$allLinesArr­ay[$i][51]','$allLinesArray[$i][52]','$a­llLinesArray[$i][53]','$allLinesArray[$i­][54]','$allLinesArray[$i][55]','$allLin­esArray[$i][56]','$allLinesArray[$i][57]­','$allLinesArray[$i][58]','$allLinesArr­ay[$i][59]','$allLinesArray[$i][60]','$a­llLinesArray[$i][61]','$allLinesArray[$i­][62]','$allLinesArray[$i][63]','$allLin­esArray[$i][64]','$allLinesArray[$i][65]­','$allLinesArray[$i][66]','$allLinesArr­ay[$i][67]','$allLinesArray[$i][140]','$­allLinesArray[$i][141]','$allLinesArray[­$i][142]','$allLinesArray[$i][143]','$al­lLinesArray[$i][144]','$allLinesArray[$i­][145]','$allLinesArray[$i][146]','$allL­inesArray[$i][147]','$allLinesArray[$i][­148]','$allLinesArray[$i][149]','$allLin­esArray[$i][150]','$allLinesArray[$i][15­1]','$allLinesArray[$i][152]','$allLines­Array[$i][153]','$allLinesArray[$i][154]­','$allLinesArray[$i][155]','$allLinesAr­ray[$i][156]','$allLinesArray[$i][157]',­'$allLinesArray[$i][158]','$allLinesArra­y[$i][159]','$allLinesArray[$i][160]','$­allLinesArray[$i][161]','$allLinesArray[­$i][162]','$allLinesArray[$i][163]','$al­lLinesArray[$i][164]','$allLinesArray[$i­][165]','$allLinesArray[$i][166]','$allL­inesArray[$i][167]','$allLinesArray[$i][­168]','$allLinesArray[$i][169]','$allLin­esArray[$i][170]','$allLinesArray[$i][17­1]','$allLinesArray[$i][172]','$allLines­Array[$i][173]','$allLinesArray[$i][174]­','$allLinesArray[$i][175]','$allLinesAr­ray[$i][176]','$allLinesArray[$i][177]',­'$allLinesArray[$i][178]','$allLinesArra­y[$i][179]','$allLinesArray[$i][180]','$­allLinesArray[$i][181]','$allLinesArray[­$i][182]','$allLinesArray[$i][183]','$al­lLinesArray[$i][184]','$allLinesArray[$i­][185]','$allLinesArray[$i][186]','$allL­inesArray[$i][187]','$allLinesArray[$i][­188]','$allLinesArray[$i][189]','$allLin­esArray[$i][190]','$allLinesArray[$i][19­1]','$allLinesArray[$i][192]','$allLines­Array[$i][193]','$allLinesArray[$i][194]­','$allLinesArray[$i][195]','$allLinesAr­ray[$i][196]','$allLinesArray[$i][197]',­'$allLinesArray[$i][198]','$allLinesArra­y[$i][199]','$allLinesArray[$i][200]','$­allLinesArray[$i][201]','$allLinesArray[­$i][202]','$allLinesArray[$i][203]','$al­lLinesArray[$i][204]')";

//echo($allRecords);
}


$insert = mysql_query("INSERT INTO placements(id,har_id,guar_num,last_name,­first_name,middle_name,ss_num,dob,sex,ma­rital_status,admin_date,admin_time,disch­arge_date,discharge_time,address_1,addre­ss_2,city,state,zip,ept_country,home_pho­ne,work_phone,mobile_phone,employer,empl­oyer_address_1,employer_address_2,employ­er_city,employer_state,employer_zip,empl­oyer_country,employer_phone,acct_base_cl­ass,fin_class,total_charges,sbo_self_pay­,sbo_bad_debt_bal,acct_bal,sbo_total_pay­ments,sbo_total_adjustments,guarantor_la­st,guarantor_first,guarantor_ss_num,guar­antor_dob,relationship,billing_address_1­,billing_address_2,billing_city,billing_­state,billing_zip,billing_country,billin­g_home_phone,billing_work_phone,ear_empl­oyer,ear_employer_address_1,ear_employer­_address_2,ear_employer_city,ear_employe­r_stateear_employer_zip,ear_employer_cou­ntry,ear_employer_phone,cvg_payor,claim_­admission_type,claim_admission_source,cl­aim_discharge_disposition,claim_occurenc­e_code_1,har_occurence_code_1,claim_occu­rence_code_2,har_occurence_code_2,claim_­occurence_code_3,har_occurence_code_3,cl­aim_occurence_code_4,har_occurence_code_­4,claim_condition_code_1,claim_condition­_code_2,claim_value_code_1,har_value_cod­e_amount_1,claim_value_code_2,har_value_­code_amount_2,har_admission_diag,har_dia­g_code_1,har_diag_code_2,har_diag_code_3­,har_diag_code_4,har_diag_code_5,har_dia­g_code_6,har_diag_code_7,har_diag_code_8­,procedure_1,procedure_date_1,procedure_­2,procedure_date_2,procedure_3,procedure­_date_3,procedure_4,procedure_date_4,pro­cedure_5,procedure_date_5,attending_phys­ician_internal_num_1,attending_physician­_internal_num_2,attending_physician_1,re­ferring_provider_21,referring_provider_2­,har_guar_id) VALUES" . $allRecords);

$result = $insert;
sendResults();
}


function sendResults(){
global $result;
echo "RESULT: " . $result;
}



what am I doing wrong here?

thanks
whispers007
whispers007
Oak Creek, WI
Post #: 7
ok.. well I think it has to do witht he data being sent in the $allRecords var.. in the INSERT command..


when I print out the $allRecords var.. I get data like this:

('',Array[1],'Array[2]','Array[3]','',­'','Array[4]','Array[5]','Array[6]','Arr­ay[7]','Array[8]','Array[9]','Array[10]'­,'Array[11]','....etc......etc.....etc..­

it seems to be putting the dataType or something?

but this prints out fine:
//echo("CHECK: " . $allLinesArray[$i][142] . "<br>");
//echo("CHECK: " . $allLinesArray[$i][2] . "<br>");


What am I doing wrong?

Is this a syntax error? with single/double quotes?
Beth Tucker L.
e3betht
Verona, WI
Post #: 34
Are you just trying to echo the array? With arrays you need to use print_r() to display all of the values.

Are you sure you are connected to the database? Those errors are caught by the database and not displayed automatically by PHP. PHP has functions to display the database errors, like:
http://www.php.net/ma...­
Beth Tucker L.
e3betht
Verona, WI
Post #: 35
Couple of yet quick things I noticed:

If you are iterating though a loop, you can use foreach instead of a for loop and the count of the array. Either works, just the foreach is a little cleaner and gives you the array items one at a time. Also, you are concatenating onto $allRecords, but I don't see that you are cleaning it at the end of each loop, so the second time it runs, it will still have the first record in it, leading to a mismatch of fields in the database query because you'll have twice as many values as field names.

I'll look at this more in a while. Right now, I'm on my phone, and it's really tough to read through the code on this tiny screen. :)
Powered by mvnForum

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy