Building an instantaneous search filer with vfp

Published on by Yousfi Benameur

     

[post 276]
vfp historically is a good searcher but with old/traditional methods as:
-pointing a folder,scan/endscan,loops,strings operations...
-vfp filer (with an external dll)-this dont worked correctly and make vfp crash without any alert (abandoned).
-gofish...
nowadays even the windows 10 search integrated in windows explorer(reappointed)  seems to be  old of age and is very slow.
there is new performent tools as "everything.exe" (free ~2.15 Mo can download  https://www.pcastuces.com/logitheque/everything.htm) making an instantaneous file search (no contents but file names only).
this is exceptional, but how to try performing an instantaneous search from vfp ? it is a good challenge !
this is an application with 2 parts :
   -first exe is embedding in windows task scheduler and have one goal: build & rebuild the mother table containing all files on any workstation.the interval of scans makes the search sensible in sens of performance.this monitores any creation,deletion,rename,moving file... on workstation by fixing an image of files in a table.
   if this monitoring is online the result can be 100% fair but there is problem of system performence (can slow the PC significately).
   -second exe performs the search on this table with a cursor.
this work stays to be performed in future. it starts with 2 exe(yrecursive.exe to install in windows scheduler (in background) and ysearch.exe the user tool).
note: can tweak the yrecursive.prg to integrate even USB disks.


      
this is a first part of my "instantaneous search" tool.
the code builds a cursor and gather all files present on hard discs of any workstation.
it detect discs and recurse in folders( an subfolders).
the cursor is copied to a permanent table allfiles.dbf in source folder.
the records in my PC are near 630.000 records with a size of 159 Moctets.
the table have a structure (path,filename,size (octets),date modified,time modified,attribute).
build a project and add a config.fpw [screen=off resource=off  safe=off]
compile an exe (as in the zip below to download)
the windows task scheduler is all indicated to run this build exe periodically.see the image below to install this task.
the task runs near 8-12 minutes to build the table.
the exe can run as standalone also on demand.
then all the goal of this yrecursive.exe task is to build the allfiles.dbf table used nextly by a second app(ysearch.exe).
A ylog.txt is built at the end of running the task( date start/end/ellapsed time...)-same info in the messagebox.

*note:for those who are not accustomed  to windows task scheduler read this:
https://www.experts-exchange.com/videos/1598/How-to-use-the-Windows-Task-Scheduler-An-Introduction.html


Click on code to select [then copy] -click outside to deselect


*1* created on friday 04 of may 2018
**save this code as yrecursive.prg
*this app have one goal: build and rebuild a same table with all files on workstation.
*this app can be scheduled from pc all 1 onde per day for ex. or maybe more less that to make a stamp of all files on workstation.
*can use another ysearch.exe application to create cursor for this recent table ,image of all files on PC to make likely an instant search
*the search is not synchronous at 100% but slightly shifted (depends on frequency to rebuild mother table)
*this app consumes  memory(see task manager).it slows the system just at starting.


If !_vfp.StartMode=0
	On Shutdown Quit
Endi

Set Defa To Addbs(Justpath(Sys(16,1)))
Local t0
t0=Datetime()

*list all discs on worksation
Declare Integer GetLogicalDrives In kernel32
Declare Integer GetDriveType In kernel32 String nDrive
Clea
Local nDrivesMask, nIndex, nShift, cDrive,x
nDrivesMask = GetLogicalDrives()
m.x= ""
nIndex = 0
Do While .T.
	nShift = Bitlshift(1, nIndex)
	cDrive = Chr(nIndex + 65) + ":"
	If Bitand(nDrivesMask, nShift) = nShift
		If GetDriveType(cDrive)=3
			m.x=m.x+ Trans(cDrive) +"\"+Chr(13)
		Endi
	Endif
	If nShift > nDrivesMask
		Exit
	Endif
	nIndex = nIndex + 1
Enddo


*List in a  curseur all files of a folder and its subfolders
Create Cursor ycurs ( cDir c(150),cFilename c(80), nSize N(10), dMod d , dtime c(5),xattrib c(5))

Local m.yrep
For i=1 To Memlines(m.x)
	m.yrep=Allt(Mline(m.x,i))
	nrep=0
	RecurseFolder( yrep )
Endfor
Sele ycurs
Index On cFilename  Tag cFilename
Set  Order To cFilename
Local m.xrec,m.xsize
m.xrec=Reccount()

Local m.xtable
Try
	Copy To "allfiles.dbf"
	m.xtable="allfiles.dbf"
Catch
	Copy To allfiles_attente.Dbf
	m.xtable="allfiles_attente.dbf"
Endtry

Close Data All
Set Compatible On
Local m.xsize,m.x
m.xsize=Fsize(m.xtable)

TEXT to m.x textmerge noshow
*Report of yrecursive.exe execution:
-Execution date :<<datetime()>>
-starting scan at  <<t0>>
-end scan at <<datetime()>>
-ellapsed time=<<(datetime()-t0)/60>> mn
-table built=<<m.xtable>>   size:<<m.xsize)>> octets.
-table records:<<m.xrec>>

ENDTEXT
Strtofile(m.x,"ylog.txt")

If File("allfiles_attente.dbf")
	Messagebox("Allfiles_attente.dbf updated -copy it manually to allfiles.dbf and erase it ",0+32+4096,"",2000)
Else
	Messagebox("Allfiles updated successfully. ",0+32+4096,"",2000)
Endi

Messagebox(m.x, 0+32+4096, '' ,4000)

If !_vfp.StartMode=0
	Quit
Else
	Return
Endi
*end of main

Function RecurseFolder( lcDir )
Local i,N, laFiles[1]
nrep=nrep+1
N = Adir( laFiles, lcDir + "*.*", "shd" )
For i = 1 To N
	Try  &&to avoid error with big discs (stack..)
		If ( Left( laFiles[i,1], 1 ) != '.' )
			If ( "D" $ laFiles[i,5] )
				RecurseFolder( lcDir + laFiles[i,1] + "\" )
			Else
				Insert Into ycurs Values( lcDir , laFiles[i,1], laFiles[i,2], laFiles[i,3],laFiles[i,4] ,laFiles[i,5])
			Endif
		Endif
	Catch
	Endtry
Endfor
Return



this is the second part of the application for "instantaneous file search".
that suppose the first already working  on PC it works fine on windows task scheduler and periodically updated with the real state of all files on workstation.
the table built allfiles.dbf ~630.000 records and size=159 Moctets in my PC
note in the recursivefolder() method i used only the SHD directories ( see Adir() function),
build a proj with this ysearch.prg add a config.fpw [screen=off resource=off safe=off] and compile ysearch.exe.put it in same folder with the table allfiles.dbf and run.
make any file search...
as a prg +config.fpw this exe is ~30 Koctets only (no debug and crypted).
some filckers appear with textbox.interactiveChange()....can disable this in checkbox on the form and use only the  button at click when typed the searched portion of file  string.
on the  ysearch.exe can dblclick on cdir or cfilename to run it with its associated application.




Click on code to select [then copy] -click outside to deselect


*2* created on friday 04 of may 2018
*save this code as ysearch.prg
*(add in the download in ysearch.prg: command1  default=.t.)

If !_vfp.StartMode=0
	On Shutdown Quit
Endi
Set Defa To Addbs(Justpath(Sys(16,1)))
Close Data All

*these 2 statements avoid showing the vfp system progressbars when searching (particulary with textbox interactiveChange...update the ysearch download).
set talk off
set notify off

*manage updates form yrecursive.exe(if allfiles.dbf in use the update is copied to allfiles_attente.dbf)
Local t1,t2
If  File("allfiles.dbf") And File("allfiles_attente.dbf")
	t1=Fdate("allfiles.dbf",1)
	t2=Fdate("allfiles_attente.dbf",1)
	If t2>t1  && allfiles_attente is most recent
		Copy File allfiles_attente.Dbf  To allfiles.Dbf
		Erase allfiles_attente.Dbf
	Else
		Erase allfiles-attente.Dbf
	Endi
Endi

If !File("allfiles.dbf")
	Messagebox("allfiles.dbf is nolonger",16+4096,'',1300)
	Return .F.
Endi

Local m.myvar
TEXT to m.myvar noshow
this app ysearch.exe uses the table allfiles.dbf built periodically in scheduler all 1 hour(to adjust).
then its slighly asynchrone and dont reflect exactly the system files state.
ENDTEXT
Messagebox(m.myvar,0+32+4096,'',3000)

Public oform
oform=Newobject("ysearch")
oform.Show
Read Events
Return
*end of main
*

*form class
Define Class ysearch As Form
	DataSession = 2
	Height = 464
	Width = 1012
	ShowWindow = 2
	AutoCenter = .T.
	Caption = "A special filer - Last allfiles.dbf update:"+trans(Fdate("allfiles.dbf",1))
	Name = "Form1"
	yic=1

	Add Object grid1 As Grid With ;
		FontSize = 8, ;
		Anchor = 15, ;
		Height = 396, ;
		Left = 12, ;
		ReadOnly = .T., ;
		RowHeight = 17, ;
		Top = 60, ;
		Width = 996, ;
		HighlightBackColor = Rgb(223,223,255), ;
		HighlightForeColor = Rgb(0,0,170), ;
		HighlightStyle = 2, ;
		Name = "Grid1"

	Add Object text1 As TextBox With ;
		Anchor = 0, ;
		Height = 25, ;
		Left = 24, ;
		Top = 6, ;
		Width = 576, ;
		Name = "Text1"
	Add Object ylab As Label With Anchor=0,BackStyle=0,Caption="dblClick cDir or filename to run the file with win associated application",Left=24,Top=36,AutoSize=.T.,ForeColor=Rgb(128,0,64),Name="ylab"

	Add Object command1 As CommandButton With ;
		Top = 6, ;
		Left = 661, ;
		Height = 25, ;
		Width = 25, ;
		Anchor = 0, ;
    default=.t.,;
		Caption = "Go", ;
		Enabled = .T., ;
		mousepointer=15,;
		fontbold=.T.,;
		Name = "Command1"
	Add Object ycom As CommandButton With Caption="All records",MousePointer=15,Left=661-40-130,Top=35,ForeColor=Rgb(128,0,64),FontBold=.T.,Name="ycom"
	Add Object Check1 As Checkbox With Value=1,Caption="enable/disable Txt InteractiveChange",AutoSize=.T.,Left= 661-40    ,Top= 35   ,Name="check1"

	Add Object Label1 As Label With ;
		Anchor = 0, ;
		BackStyle = 0, ;
		Caption = "", ;
		Height = 25, ;
		Left = 736, ;
		Top = 5, ;
		Width = 241, ;
		ForeColor = Rgb(255,0,0), ;
		Name = "Label1"

	Procedure ycom.Click
	Thisform.Label1.Caption=""
	Sele * From allfiles.Dbf  Into Cursor ycurs
	Sele ycurs
	With Thisform.grid1
		.RecordSource=""
		.RecordSource="ycurs"
		.RecordSourceType=1
		.column2.FontBold=.T.
		.column1.Width=0.4*.Parent.Width -20
		.column2.Width=0.2*.Parent.Width -20
		.column3.Width=0.1*.Parent.Width -20
		.column4.Width=0.1*.Parent.Width -20
		.column5.Width=0.1*.Parent.Width -20
		.column6.Width=0.1*.Parent.Width -20
		.SetAll("DynamicBackColor","IIF(MOD(RECNO(), 2)=0,rgb(220,220,220)  , rgb(255,255,255))", "Column")
		Locate
		.Refresh
		.Parent.Label1.Caption=Trans(Reccount())+" occurs"
		For i=1 To .ColumnCount
			Bindevent(.Columns(i).text1,"rightclick",Thisform,"yrmenu")
		Endfor
		Bindevent(.column1.text1,"dblclick",Thisform,"yrun")
		Bindevent(.column2.text1,"dblclick",Thisform,"yrun")
	Endwith
	Endproc

	Procedure  Check1.InteractiveChange
	Thisform.command1.Enabled=Iif(This.Value=1,1,0)
	If This.Value=1
		Thisform.yic=1
	Else
		Thisform.yic=0
	Endi

	Endproc

	Procedure yrmenu
	Define Popup raccourci SHORTCUT Relative From Mrow(),Mcol()
	Define Bar _Med_slcta Of raccourci Prompt "Sélec\<tionner tout" ;
		KEY CTRL+A, "Ctrl+A" ;
		MESSAGE "Sélectionne tout le texte ou tous les éléments de la fenêtre active"
	Define Bar _Med_cut Of raccourci Prompt "\<Couper" ;
		KEY CTRL+X, "Ctrl+X" ;
		MESSAGE "Enlève la sélection et la place dans le Presse-papiers"
	Define Bar _Med_paste Of raccourci Prompt "C\<oller" ;
		KEY CTRL+V, "Ctrl+V" ;
		MESSAGE "Place le contenu du Presse-papiers au point d'insertion"
	Define Bar _Med_copy Of raccourci Prompt "Co\<pier" ;
		KEY CTRL+C, "Ctrl+C" ;
		MESSAGE "Copie la sélection et la place dans le Presse-papiers"
	Define Bar _Med_undo Of raccourci Prompt "\<Annuler" ;
		KEY CTRL+Z, "Ctrl+Z" ;
		MESSAGE "Annule la dernière modification"
	Define Bar _Med_redo Of raccourci Prompt "\<Rétablir" ;
		KEY CTRL+R, "Ctrl+R" ;
		MESSAGE "Rétablit la dernière opération annulée"
	Activate Popup raccourci
	Endproc

	Procedure yrun
	Sele ycurs
	result = ShellExecute(0, "open", Allt(cdir)+Allt(cfilename),"","",1)
	If result<=32
		Messagebox("An error was occured with window associated application",16+4096,'',1300)
	Endi
	Endproc

	Procedure Destroy
	Close Data All
	Clea Events
	Endproc

	Procedure Resize
	With Thisform.grid1
		.column1.Width=0.4*.Parent.Width -20
		.column2.Width=0.2*.Parent.Width -20
		.column3.Width=0.1*.Parent.Width -20
		.column4.Width=0.1*.Parent.Width -20
		.column5.Width=0.1*.Parent.Width -20
		.column6.Width=0.1*.Parent.Width -20
	Endwith
	Endproc

	Procedure Load
&&shellexecute
	Declare Integer ShellExecute In SHELL32.Dll Integer nWinHandle,;
		STRING cOperation,;
		STRING cFileName,;
		STRING cParameters,;
		STRING cDirectory,;
		INTEGER nShowWindow
	Close Data All
	Sele * From allfiles.Dbf  Into Cursor ycurs
	Endproc

	Procedure Init
	Thisform.text1.SetFocus
	Thisform.Label1.Caption=Trans(Reccount())+" records"
	Endproc

	Procedure grid1.Init
	With This
		.RecordSource="ycurs"
		.RecordSourceType=1
		.GridLines=0
		.DeleteMark=.F.
		.column2.FontBold=.T.
		.column1.Width=0.4*.Parent.Width -20
		.column2.Width=0.2*.Parent.Width -20
		.column3.Width=0.1*.Parent.Width -20
		.column4.Width=0.1*.Parent.Width -20
		.column5.Width=0.1*.Parent.Width -20
		.column6.Width=0.1*.Parent.Width -20
		.SetAll("DynamicBackColor","IIF(MOD(RECNO(), 2)=0, rgb(220,220,220)  , rgb(255,255,255))", "Column")
		Locate
		.Refresh

		For i=1 To .ColumnCount
			Bindevent(.Columns(i).text1,"rightclick",Thisform,"yrmenu")
		Endfor
		Bindevent(.column1.text1,"dblclick",Thisform,"yrun")
		Bindevent(.column2.text1,"dblclick",Thisform,"yrun")
	Endwith
	Endproc

	Procedure text1.InteractiveChange
	If Thisform.yic=1
		Return .F.
	Endi

	Thisform.Label1.Caption=""
	Sele * From allfiles.Dbf Where Lower(Allt(Thisform.text1.Value)) $ Lower(Allt(cfilename)) Into Cursor ycurs
	Sele ycurs
	With Thisform.grid1
		.RecordSource=""
		.RecordSource="ycurs"
		.RecordSourceType=1
		.column2.FontBold=.T.
		.column1.Width=0.4*.Parent.Width -20
		.column2.Width=0.2*.Parent.Width -20
		.column3.Width=0.1*.Parent.Width -20
		.column4.Width=0.1*.Parent.Width -20
		.column5.Width=0.1*.Parent.Width -20
		.column6.Width=0.1*.Parent.Width -20
		.SetAll("DynamicBackColor","IIF(MOD(RECNO(), 2)=0,rgb(220,220,220)  , rgb(255,255,255))", "Column")
		Locate
		.Refresh
		.Parent.Label1.Caption=Trans(Reccount())+" occurs"
		For i=1 To .ColumnCount
			Bindevent(.Columns(i).text1,"rightclick",Thisform,"yrmenu")
		Endfor
		Bindevent(.column1.text1,"dblclick",Thisform,"yrun")
		Bindevent(.column2.text1,"dblclick",Thisform,"yrun")
	Endwith
	Endproc

	Procedure text1.RightClick
	Thisform.yrmenu()
	Endproc

	Procedure command1.Click
	Thisform.Label1.Caption=""
	Sele * From allfiles.Dbf Where Lower(Allt(Thisform.text1.Value)) $ Lower(Allt(cfilename)) Into Cursor ycurs
	Sele ycurs
	With Thisform.grid1
		.RecordSource=""
		.RecordSource="ycurs"
		.RecordSourceType=1
		.column2.FontBold=.T.
		.column1.Width=0.4*.Parent.Width -20
		.column2.Width=0.2*.Parent.Width -20
		.column3.Width=0.1*.Parent.Width -20
		.column4.Width=0.1*.Parent.Width -20
		.column5.Width=0.1*.Parent.Width -20
		.column6.Width=0.1*.Parent.Width -20
		.SetAll("DynamicBackColor","IIF(MOD(RECNO(), 2)=0,rgb(220,220,220)  , rgb(255,255,255))", "Column")
		Locate
		.Refresh
		.Parent.Label1.Caption=Trans(Reccount())+" occurs"
		For i=1 To .ColumnCount
			Bindevent(.Columns(i).text1,"rightclick",Thisform,"yrmenu")
		Endfor

		Bindevent(.column1.text1,"dblclick",Thisform,"yrun")
		Bindevent(.column2.text1,"dblclick",Thisform,"yrun")
	Endwith
	Endproc
Enddefine
*
*-- EndDefine: ysearch


if want to integrate directories also in search simply replace in sql command this (its slows slightly the processus):

Sele * From allfiles.Dbf Where Lower(Allt(Thisform.text1.Value)) $ Lower(Allt(cfilename)) or Lower(Allt(Thisform.text1.Value)) $ cDir Into Cursor ycurs

this is how to install the yrecursive.exe in  windows task scheduler (must have an account and login)-(frequency can be at 9:30 in all days for ex).

this is how to install the yrecursive.exe in windows task scheduler (must have an account and login)-(frequency can be at 9:30 in all days for ex).


*comments:
in the textbox type for ex:
  -any portion of the file name (can be anywhere in the string filename)
  -any extension of the file name(ico,jpg,.png,.h,.prg,.exe,html....)

*note can extend the search to folders names,file size,date modified,attributes....this is to do in future by anyone.this is a first work in this tool.
*can enable textbox interactiveChange (with some flickers here if talk=on,notify=on) or better enable button (simple click to search).
*dblclick on filename or folder to launch execution of the fullname with shellexecute API.
*of course table allfiles.dbf must be in source folder (created initially by yrcursive.exe with scheduler or directly with exe).
*naturally ,this kind of app is built for all hard disks on a workstation.it consumes times to gather all files in a table.for a folder old method are suffisent and nearly "instantaneaous".
*can add as exercice the file icons on grid (see previous post on this subject).
*the last update of the mother table (allfiles.dbf) is in the form.caption.


the code above *1* and  *2* are updated in the download below.
the code above *1* and  *2* are updated in the download below.
the code above *1* and  *2* are updated in the download below.
the code above *1* and  *2* are updated in the download below.
the code above *1* and  *2* are updated in the download below.

the code above *1* and *2* are updated in the download below.

Click on code to select [then copy] -click outside to deselect

*3* created on friday 04 of may 2018
*three vfp methods to launch programmatly the windows task scheduler
1.
local oshell
oshell=newObject("wscript.shell")
oshell.run("%SystemRoot%\system32\taskschd.msc /s")
oshell=null

-----------------------------------------------------
2.
&&shellexecute
DECLARE INTEGER ShellExecute IN SHELL32.DLL INTEGER nWinHandle,;
                                            STRING cOperation,;
                                            STRING cFileName,;
                                            STRING cParameters,;
                                            STRING cDirectory,;
                                            INTEGER nShowWindow
result = ShellExecute(0, "open","c:\windows\system32\taskschd.msc" ," /s","",1)


-------------------------------------------------------
3.
&&shellexecute
DECLARE INTEGER ShellExecute IN SHELL32.DLL INTEGER nWinHandle,;
                                            STRING cOperation,;
                                            STRING cFileName,;
                                            STRING cParameters,;
                                            STRING cDirectory,;
                                            INTEGER nShowWindow
result = ShellExecute(0, "open","control.exe" ," schedtasks","",1)

*the windows tasks scheduler can controlled programmatly with command lines as well as mnually.
*read this reference:https://msdn.microsoft.com/en-us/library/windows/desktop/bb736357(v=vs.85).aspx



                     

Yousfi Benameur


Important:All Codes above are tested on VFP9SP2 & windows 10 pro 64 bits version 1709(fall creator) & IE11 emulation. Navigator: firefox - screen:32 pouces.

To be informed of the latest articles, subscribe:
Comment on this post
A
Liking the sharing of code - it is always a good thing in my book as life shouldnt be full of hurdles :)
Reply