I have a simple code that loops through a text file to fix a few things in it and I use mostly dictionaries to find and handle the adjustment. However, with larger files (about 30K lines) it seems to get exponentially slower in a specific line of a loop. This line shouldn't be so slow because it just has to access an element specified and alter value. Anyone knows why this is happening ?
I can't figure out why that is happening and how to go around without a huge performance loss.
For i = 0 To tags.Count - 1
If tags.Items(i)(2) = 0 Then 'This is the slow part. I need the validation for my case usage.
linhas(CStr(i)) = 1
End If
Next i
This is the full code simplified:
Sub ajusta_ofx_teste() 'macro para converter arquivos ofx em xml
Dim my_file As Integer
Dim text_line As String
Dim file_name As String
Dim i As Long
Dim linhas As New Scripting.Dictionary
Dim tags As New Scripting.Dictionary
Dim fso As New Scripting.FileSystemObject
Dim nfso As New Scripting.FileSystemObject
Dim dados_tag(4) As Variant
strofx = Application.GetOpenFilename(MultiSelect:=False)
'Open file_name For Input As #my_file
Set arquivo = fso.OpenTextFile(strofx, ForReading)
L = 10000
f = 0
i = 0 'Define o número de linhas com Tag
loops = 0 'Define a linha do arquivo sendo lida
ntags = 0
achou = 0
While Not arquivo.AtEndOfStream
'Line Input #my_file, text_line
text_line = arquivo.ReadLine
If i > 100000 Then Stop
x = InStr(1, text_line, "<") 'Marca o início do Tag
'Save only lines with Tag in "tag" dictionary.
If x > 0 Then
linhas.Add CStr(i), text_line
If Mid(text_line, x + 1, 1) <> "/" Then
ntags = ntags + 1
fim_tag_i = InStr(1, text_line, ">")
dados_tag(0) = i 'Tag line number
dados_tag(1) = Mid(text_line, x, fim_tag_i - x + 1) 'Tag name
dados_tag(2) = 0 'Registry used after - I removed the part of the code that fills it depending on the tag.
dados_tag(3) = 0
chave = Format(ntags, "000000") & dados_tag(1) 'chave = "Key"
tags.Add chave, dados_tag
End If
i = i + 1
End If
loops = loops + 1
If loops > 100000 Then Stop 'saida de emergencia do loop
Wend
nlinhas = i
arquivo.Close
For i = 0 To tags.Count - 1
If tags.Items(i)(2) = 0 Then 'This is the slow part. I need the validation for my case usage.
linhas(CStr(i)) = 1
End If
Next i
Set fso = Nothing
Set nfso = Nothing
End Sub
The text file can be downloaded here: test.ofx
