Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add PageSize option to more cmdlets #4141

Open
sg1888 opened this issue Aug 7, 2024 · 2 comments
Open

[FEATURE] Add PageSize option to more cmdlets #4141

sg1888 opened this issue Aug 7, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@sg1888
Copy link

sg1888 commented Aug 7, 2024

Is your feature request related to a problem? Please describe.
I would like to have -PageSize as an option for the following cmdlets: Get-PnPFileInFolder, Get-PnPFolderItem, and Get-PnPFolderInFolder. Right now, PageSize is not supported.

I have a large Sharepoint library with > 4 million files. Right now I have to use Get-PnPListItem, which loads the entire site. This is incredibly slow and memory intensive. I tried using Get-PnPListItem -List "Documents" -FolderServerRelativeURL "/Sites/Url/", but it doesn't seem to work when PageSize is specified.

Describe the solution you'd like
I would like to be able to specify PageSize for the aforementioned cmdlets so that they can be used on large folders and subfolders.

Describe alternatives you've considered
I've considered buying 256GB of RAM and carving out 2-3 weeks to let the script work, but there has to be a better way.

@sg1888 sg1888 added the enhancement New feature or request label Aug 7, 2024
@jackpoz
Copy link
Contributor

jackpoz commented Aug 8, 2024

To avoid loading the whole list/library in memory, use the -ScriptBlock parameter of Get-PnPListItem

The script block to run after every page request.

It accepts script block like { Param($items) } where $items contains as many items as specified in -PageSize .
I was able to process 1.5 M files in a library this way without any RAM issues.

@sg1888
Copy link
Author

sg1888 commented Aug 9, 2024

To avoid loading the whole list/library in memory, use the -ScriptBlock parameter of Get-PnPListItem

The script block to run after every page request.

It accepts script block like { Param($items) } where $items contains as many items as specified in -PageSize . I was able to process 1.5 M files in a library this way without any RAM issues.

Thanks. I didn't think of using ScriptBlock - I will try that. I do think that adding PageSize to more cmdlets would be helpful as scriptblocks are a bit kludgy, especially if you need to reference other variables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants